Posted on | Edited on | In

Hello everyone… As you all know that currently, my hometown Wuhan, China is suffering the MOST SERIOUS infective disease - Corona Virus in her history. My parents are still in this locked city. People have been suffering the threaten from the death for over 10 days. I would have to write something, and WISH ALL MY BEST to my beloved parents, and all my country fellows who are still in Wuhan.

In this blog, all data are obtained from WHO Corona Virus Reports.
And, all plots are generated by altair.

Today, I can ONLY make that much…

Confirmed In China

Death Over World

God bless Dr. Li WenLiang, and shame on rubbish in Chinese government.

Updated on:

  • February 10, 2020
  • February 14, 2020

So far, the data is lack of a lot of info including age, gender, etc. Let’s keep an eye on Chinese Corona Virus**.

Posted on | In , 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
➜  binaries sudo ./hikey_idt -c config
Config name: config
0: Image: ./sec_usb_xloader.img Downalod Address: 0x22000
1: Image: ./sec_usb_xloader2.img Downalod Address: 0x60049000
2: Image: ./l-loader.bin Downalod Address: 0x16800000
Serial port open successfully!
Start downloading ./sec_usb_xloader.img@0x22000...
file total size 122880
downlaod address 0x22000
Finish downloading
Start downloading ./sec_usb_xloader2.img@0x60049000...
file total size 40960
downlaod address 0x60049000
Finish downloading
Start downloading ./l-loader.bin@0x16800000...
file total size 1179648
downlaod address 0x16800000
Finish downloading
➜  binaries cd ../   
➜  Hikey970-shunya-desktop-image-0.1 sudo ./binaries/recovery-flash.sh 
./binaries/recovery-flash.sh: 7: [: unexpected operator
Config name: ....../Hikey970-shunya-desktop-image-0.1/binaries/config
....../Hikey970-shunya-desktop-image-0.1/binaries/hikey_idt: option requires an argument -- 'p'
Usage: hikey_idt -c config
Flashing ptable 
Sending 'ptable' (24 KB)                           OKAY [  0.001s]
Writing 'ptable'                                   OKAY [  0.005s]
Finished. Total time: 0.007s
Sending 'xloader' (164 KB)                         OKAY [  0.005s]
Writing 'xloader'                                  OKAY [  0.009s]
Finished. Total time: 0.014s
Sending 'fastboot' (1152 KB)                       OKAY [  0.034s]
Writing 'fastboot'                                 OKAY [  0.056s]
Finished. Total time: 0.091s
Sending 'fip' (1224 KB)                            OKAY [  0.036s]
Writing 'fip'                                      OKAY [  0.058s]
Finished. Total time: 0.098s
Sending 'boot' (65536 KB)                          OKAY [  2.143s]
Writing 'boot'                                     OKAY [  0.418s]
Finished. Total time: 2.590s
Please be patient...
Sending sparse 'userdata' 1/6 (131068 KB)          OKAY [  5.894s]
Writing 'userdata'                                 OKAY [  0.674s]
Sending sparse 'userdata' 2/6 (131068 KB)          OKAY [  4.250s]
Writing 'userdata'                                 OKAY [  0.740s]
Sending sparse 'userdata' 3/6 (131068 KB)          OKAY [  4.039s]
Writing 'userdata'                                 OKAY [  0.687s]
Sending sparse 'userdata' 4/6 (131068 KB)          OKAY [  4.029s]
Writing 'userdata'                                 OKAY [  0.670s]
Sending sparse 'userdata' 5/6 (131068 KB)          OKAY [  3.526s]
Writing 'userdata'                                 OKAY [  0.672s]
Sending sparse 'userdata' 6/6 (66608 KB)           OKAY [  1.827s]
Writing 'userdata'                                 OKAY [  0.346s]
Finished. Total time: 27.425s

Posted on | In ,

What a sunny day after the FIRST snow in this winter. Let me show you 3 pictures in the first row, and 3 videos in the second row. We need to enjoy both R&D and life…

Green Timers Lake 1 Green Timers Lake 2 Green Timers Park
Green Timers Lake 1 Green Timers Lake 2 Green Timers Park
A Pair of Swans A Group of Ducks A Little Stream In The Snow

After a brief break, I started investigating Khadas VIM3 again.

1. About Khadas VIM3

Khadas VIM3 is a super computer based on Amlogic A311D. Before we start, let’s carry out several simple comparisons.

1.1 Raspberry Pi 4 Model B vs. Khadas VIM3 vs. Jetson Nano Developer Kit

Please refer to hackerboards comparisons.

1.2 Amlogic A311D & S922X-B vs. Rockchip RK3399 (Pro) vs. Amlogic S912

Please refer to:

2. Ubuntu XFCE for Khadas VIM3

Let’s try out Ubuntu XFCE on Khadas VIM3.

2.1 Ubuntu XFCE Installation

As of today, I just downloaded VIM3_Ubuntu-xfce-bionic_Linux-5.5-rc2_arm64_SD-USB_V0.8.2-20200103.img, and flash it onto a TF card as:

1
2
3
4
➜  khadas sudo dd bs=4M if=VIM3_Ubuntu-xfce-bionic_Linux-5.5-rc2_arm64_SD-USB_V0.8.2-20200103.img of=/dev/mmcblk0 conv=fsync
1093+0 records in
1093+0 records out
4584374272 bytes (4.6 GB, 4.3 GiB) copied, 206.586 s, 22.2 MB/s

2.2 SSH Into Khadas VIM3

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
➜  ~ ssh khadas@192.168.1.95
The authenticity of host '192.168.1.95 (192.168.1.95)' can't be established.
ECDSA key fingerprint is SHA256:Q59XrIX7bSWsphZCpgHBSnVH5ETgCY9iLfDEuvRKtOw.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.95' (ECDSA) to the list of known hosts.
khadas@192.168.1.95's password: 

Welcome to Fenix 0.8.2 Ubuntu 18.04.3 LTS Linux 5.5.0-rc2  
 _  ___               _            __     _____ __  __ _____ 
| |/ / |__   __ _  __| | __ _ ___  \ \   / /_ _|  \/  |___ / 
| ' /| '_ \ / _` |/ _` |/ _` / __|  \ \ / / | || |\/| | |_ \ 
| . \| | | | (_| | (_| | (_| \__ \   \ V /  | || |  | |___) |
|_|\_\_| |_|\__,_|\__,_|\__,_|___/    \_/  |___|_|  |_|____/ 
                                                             

 * Website:        https://www.khadas.com
 * Documentation:  https://docs.khadas.com
 * Forum:          https://forum.khadas.com
khadas@Khadas:~$ uname -a
Linux Khadas 5.5.0-rc2 #0.8.2 SMP PREEMPT Fri Jan 3 09:48:06 CST 2020 aarch64 aarch64 aarch64 GNU/Linux
khadas@Khadas:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.3 LTS
Release:	18.04
Codename:	bionic
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
......
Installing Amlogic flash-tool...

===============================================

Host PC: Ubuntu 18.04

===============================================

Installing USB rules...
Installing flash-tool...
Done!

Installing Rockchip flash-tool...

===============================================

Host PC: Ubuntu 18.04

===============================================

Installing USB rules...
Installing flash-tool...
Installing configuration...
Done!

Installing ToneBoard burn-tool...

===============================================

Host PC: Ubuntu 18.04

===============================================

Installing USB rules...
Installing toneboard-burn-tool...
Done!
Installing Khadas burn-tool...
Done!

3. OpenVINO Installation

Let’s just follow this Intel official documentation Arm* 64 Single Board Computers and the Intel® Neural Compute Stick 2 (Intel® NCS 2).

Posted on | Edited on | In ,

First snow in 2020. Actually, it is ALSO the FIRST snow for the winter from 2019 to 2020.

First Snow 1 First Snow 2 First Snow 3
First Snow 1 First Snow 2 First Snow 3

Both my son and the Chinese New Year are coming. Let’s start the mode of celebrating. Today, I’m going to do the hotpot.

Hotpot 1 Hotpot 2 Hotpot 3
Hotpot 1 Hotpot 2 Hotpot 3

It looks in 1-day time, everybody is doing the edge computing. Today, we’re going to have some fun of Google Coral.

1. Google Coral USB Accelerator

Image cited from Coral official website.

Google Coral USB Accelerator

To try out Google Coral USB Accelerator is comparitively simple. The ONLY thing to do is just to follow Google Doc - Get started with the USB Accelerator. Anyway, let’s test it out with the following commands.

Make sure we are able to list the device.

1
2
3
➜  classification git:(master) ✗ lsusb
Bus 002 Device 003: ID 1a6e:089a Global Unichip Corp.
...

We then run the example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
➜  classification git:(master) ✗ python3 classify_image.py \
--model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
--labels models/inat_bird_labels.txt \
--input images/parrot.jpg
INFO: Initialized TensorFlow Lite runtime.
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
12.6ms
3.9ms
4.3ms
4.2ms
4.1ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.76172

BTW, I’m going to discuss Google Coral TPU, Intel Movidius VPU and Cambricon NPU which has been adopted in HuaWei Hikey 970 and Rockchip 3399 Pro, sooner or later. Just keep an eye on my blog.

2. Google Coral Dev Board

In the following, we’re going to disscuss Google Coral Dev Board more. Image cited from Coral official website.

Google Coral Dev Board

2.1 Mendel Installation

2.1.1 Mendel Linux Preparation

Google Corel Mendel Linux can be downloaded from https://coral.ai/software/. In our case, we are going to try Mendel Linux 4.0.

2.1.2 Connect Dev Board

On the host, we should be able to see:

1
2
3
4
5
6
7
8
9
➜  mendel-enterprise-day-13 lsusb
......
Bus 001 Device 020: ID 10c4:ea70 Cygnal Integrated Products, Inc. CP210x UART Bridge
......
➜  mendel-enterprise-day-13 dmesg | grep ttyUSB
[45021.091322] usb 1-8: cp210x converter now attached to ttyUSB0
[45021.092681] usb 1-8: cp210x converter now attached to ttyUSB1
➜  mendel-enterprise-day-13 screen /dev/ttyUSB0 115200
.....

Now what you see is a black screen. After having connected the Type C power cable, we should be able to see:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
power_bd71837_init
pmic debug: name=BD71837
Board id: 6
Training PASS
Normal Boot
Trying to boot from MMC1


U-Boot 2017.03 (Dec 06 2018 - 19:26:58 +0000)

CPU:   Freescale i.MX8MQ rev2.0 1500 MHz (running at 1000 MHz)
CPU:   Commercial temperature grade (0C to 95C)FSL_SDHC: 0, FSL_SDHC: 1
*** Warning - bad CRC, using default environment

flash target is MMC:0
Net:   
Warning: ethernet@30be0000 using MAC address from ROM
eth0: ethernet@30be0000

u-boot=>

That is what’s printed from Google Coral Dev Board. After having connected the Type C OTG cable, we should be able to see on the host:

1
2
➜  mendel-enterprise-day-13 fastboot devices
101989d6f32efb39	fastboot

2.1.3 Flash Corel Dev Board

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
➜  mendel-enterprise-day-13 ls
boot_arm64.img  flash.sh  partition-table-16gb.img  partition-table-64gb.img  partition-table-8gb.img  README  recovery.img  rootfs_arm64.img  u-boot.imx
➜  mendel-enterprise-day-13 bash flash.sh
Sending 'bootloader0' (991 KB)                     OKAY [  0.055s]
Writing 'bootloader0'                              OKAY [  0.190s]
Finished. Total time: 0.266s
Rebooting into bootloader                          OKAY [  0.024s]
Finished. Total time: 0.125s
Sending 'gpt' (33 KB)                              OKAY [  0.018s]
Writing 'gpt'                                      OKAY [  0.309s]
Finished. Total time: 0.346s
Rebooting into bootloader                          OKAY [  0.022s]
Finished. Total time: 0.122s
Erasing 'misc'                                     OKAY [  0.069s]
Finished. Total time: 0.079s
Sending 'boot' (131072 KB)                         OKAY [  5.321s]
Writing 'boot'                                     OKAY [  3.632s]
Finished. Total time: 8.972s
Sending sparse 'rootfs' 1/4 (368422 KB)            OKAY [ 14.792s]
Writing 'rootfs'                                   OKAY [ 36.191s]
Sending sparse 'rootfs' 2/4 (408501 KB)            OKAY [ 16.646s]
Writing 'rootfs'                                   OKAY [ 18.944s]
Sending sparse 'rootfs' 3/4 (389107 KB)            OKAY [ 15.881s]
Writing 'rootfs'                                   OKAY [ 37.021s]
Sending sparse 'rootfs' 4/4 (231325 KB)            OKAY [  9.482s]
Writing 'rootfs'                                   OKAY [ 65.204s]
Finished. Total time: 214.205s
Rebooting                                          OKAY [  0.005s]
Finished. Total time: 0.105s
➜  mendel-enterprise-day-13 mdt devices
wishful-orange		(192.168.101.2)

You will be able to see Google Coral Dev Board is NOW connected. If you don’t see the EXPECTED output wishful-orange (192.168.101.2), just plug out and plug in the Type C power cable again.

2.1.4 Boot Mendel

Unfortunately, mdt tool does NOT work properly.

1
2
3
4
5
6
7
8
9
10
11
➜  mendel-enterprise-day-13 mdt shell
Waiting for a device...
Connecting to wishful-orange at 192.168.101.2
Key not present on wishful-orange -- pushing

It looks like you're trying to connect to a device that isn't connected
to your workstation via USB and doesn't have the SSH key this MDT generated.
To connect with `mdt shell` you will need to first connect to your device
ONLY via USB.

Cowardly refusing to attempt to push a key to a public machine.

This bug has been clarified on StackOverflow. By modifying file vim $HOME/.local/lib/python3.6/site-packages/mdt/sshclient.py line 86, from if not self.address.startswith('192.168.100'): to if not self.address.startswith('192.168.10'):, problem solved.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
➜  mendel-enterprise-day-13 mdt shell
Waiting for a device...
Connecting to wishful-orange at 192.168.101.2
Key not present on wishful-orange -- pushing
Linux wishful-orange 4.14.98-imx #1 SMP PREEMPT Fri Nov 8 23:28:21 UTC 2019 aarch64

The programs included with the Mendel GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Mendel GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Feb 14 10:12:02 2019
mendel@wishful-orange:~$ ls
mendel@wishful-orange:~$ pwd
/home/mendel
mendel@wishful-orange:~$ uname -a
Linux wishful-orange 4.14.98-imx #1 SMP PREEMPT Fri Nov 8 23:28:21 UTC 2019 aarch64 GNU/Linux
mendel@wishful-orange:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Mendel
Description:	Mendel GNU/Linux 4 (Day)
Release:	10.0
Codename:	day
mendel@wishful-orange:~$ ip -c address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 7c:d9:5c:b1:fa:cc brd ff:ff:ff:ff:ff:ff
3: wlan0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 3000
    link/ether 7c:d9:5c:b1:fa:cd brd ff:ff:ff:ff:ff:ff
4: p2p0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 3000
    link/ether 00:0a:f5:89:89:81 brd ff:ff:ff:ff:ff:ff
5: usb0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether 02:22:78:0d:f6:df brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.2/24 brd 192.168.100.255 scope global noprefixroute usb0
       valid_lft forever preferred_lft forever
    inet6 fe80::cc6d:b3d4:f07e:eed1/64 scope link tentative noprefixroute 
       valid_lft forever preferred_lft forever
6: usb1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 02:22:78:0d:f6:de brd ff:ff:ff:ff:ff:ff
    inet 192.168.101.2/24 brd 192.168.101.255 scope global noprefixroute usb1
       valid_lft forever preferred_lft forever
    inet6 fe80::5bf4:c217:d9c9:859c/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
mendel@wishful-orange:~$

After activate the Internet by nmtui, we can NOW clearly see the wlan0 IP is automatically allocated.

1
2
3
4
5
6
7
8
9
10
11
mendel@wishful-orange:~$ ip -c address
......
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 3000
    link/ether 7c:d9:5c:b1:fa:cd brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.110/24 brd 192.168.1.255 scope global dynamic noprefixroute wlan0
       valid_lft 86367sec preferred_lft 86367sec
    inet6 2001:569:7e6e:dc00:d1c4:697a:f60e:b5a4/64 scope global dynamic noprefixroute 
       valid_lft 7468sec preferred_lft 7168sec
    inet6 fe80::e10b:9dc6:60c4:b91b/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
......

Of course, we can setup a static IP for this particular Google Coral Dev Board afterwards.

2.1.5 SSH into Mendel

In order to SSH into Mendel and connect remotely, we need to do Connect to a board’s shell on the host computer. You MUST pushkey before you can ssh into the board via the Internet IP instead of the virtual IP via USB, say 192.168.100.2 or 192.168.101.2.

2.2 Demonstration

2.2.1 edgetpu_demo --device & edgetpu_demo --stream

Let’s ignore edgetpu_demo --device for I ALMOST NEVER work with a GUI mode. The demo video is on my youtube channel, please refer to:

On console, it just displays as:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
mendel@deft-orange:~$ edgetpu_demo --stream
Press 'q' to quit.
Press 'n' to switch between models.

(edgetpu_detect_server:9991): Gtk-WARNING **: 07:56:57.725: Locale not supported by C library.
	Using the fallback 'C' locale.
INFO:edgetpuvision.streaming.server:Listening on ports tcp: 4665, web: 4664, annexb: 4666
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:37536
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:37538
INFO:edgetpuvision.streaming.server:[192.168.1.200:37536] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:37536] Tx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.1.200:37536] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.1.200:37536] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:37540
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:37542
INFO:edgetpuvision.streaming.server:Number of active clients: 3
INFO:edgetpuvision.streaming.server:[192.168.1.200:37538] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:37540] Rx thread finished
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:37544
INFO:edgetpuvision.streaming.server:[192.168.1.200:37538] Tx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:37542] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:37542] Tx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 4
......

2.2.2 Classification

1
2
3
4
5
6
7
8
9
10
11
12
13
14
mendel@wishful-orange:~/Downloads/tflite/python/examples/classification$ python3 classify_image.py \
> --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
> --labels models/inat_bird_labels.txt \
> --input images/parrot.jpg
INFO: Initialized TensorFlow Lite runtime.
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
14.2ms
2.7ms
2.6ms
2.6ms
2.9ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.76562

2.2.3 Camera

2.2.3.1 Google Coral camera

The Google Coral camera can be detected as a video device:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
mendel@wishful-orange:~$ v4l2-ctl --list-formats-ext --device /dev/video0
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'YUYV' (YUYV 4:2:2)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 720x480
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 2592x1944
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 0x0

2.2.3.2 Face Detection Using Google TPU

My youtube real-time face detection video clearly shows Google TPU is seriously powerful.

On console, it displays:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
mendel@deft-orange:~$ edgetpu_detect_server \
> --model ${DEMO_FILES}/mobilenet_ssd_v2_face_quant_postprocess_edgetpu.tflite

(edgetpu_detect_server:4081): Gtk-WARNING **: 10:40:45.436: Locale not supported by C library.
	Using the fallback 'C' locale.
INFO:edgetpuvision.streaming.server:Listening on ports tcp: 4665, web: 4664, annexb: 4666
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:33950
INFO:edgetpuvision.streaming.server:[192.168.1.200:33950] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:33950] Tx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:[192.168.1.200:33950] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.1.200:33950] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 0
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:33952
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:[192.168.1.200:33952] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:33952] Tx thread finished
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:33954
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.1.200:33952] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.1.200:33954] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:33952] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
......

Posted on | Edited on | In ,

Merry Christmas and happy new year everybody. I’ve been back to Vancouver for several days. These days, I’m updating this blog FIRSTLY written in September 2019. 2020 is coming, and we’re getting 1 year older. A kind of sad hum?

Okay… No matter what, let’s enjoy the song first: WE ARE YOUNG. Today, I joined Free Software Foundation and start my journey of supporting Open Source Software BY CASH. For me, it’s not about poverity or richness. It’s ALL about FAITH.

To write something about Raspberry Pi is to say GOOD BYE to my Raspberry Pi 3B, and WELCOME Raspberry Pi 4 at the same time. Our target today is to build an AI edge computing end as the following Youtube video:

1. About Raspberry Pi 4

1.1 Raspberry Pi 4 vs. Raspberry Pi 3B+

Before we start, let’s carry out a simple comparison between Raspberry Pi 4 and Raspberry Pi 3B+.

1.2 Raspbian Installation

1
2
3
4
➜  raspbian sudo dd bs=4M if=2019-09-26-raspbian-buster.img of=/dev/mmcblk0 conv=fsync 
913+0 records in
913+0 records out
3829399552 bytes (3.8 GB, 3.6 GiB) copied, 144.192 s, 26.6 MB/s

1.3 BCM2711 is detected as BCM2835

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
pi@raspberrypi:~ $ cat /proc/cpuinfo
processor	: 0
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 108.00
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 1
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 108.00
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 2
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 108.00
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 3
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 108.00
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

Hardware	: BCM2835
Revision	: c03111
Serial		: 100000006c0c9b01
Model		: Raspberry Pi 4 Model B Rev 1.1

This issue seems to be a well-known bug. Raspberry Pi 4’s specification can be retrieved from The MagPi Magazine. More details about the development history of Raspberry Pi can be found on Wikipedia.

2. Movidius Neural Compute Stick on Raspberry Pi 4

Then, we just follow the following 2 blogs Run NCS Applications on Raspberry Pi and Adding AI to the Raspberry Pi with the Movidius Neural Compute Stick to test out Intel Movidius Neural Compute Stick 2:

Intel Movidius Neural Compute Stick 2 Intel Movidius Neural Compute Stick 1
Intel Movidius Neural Compute Stick 2 Intel Movidius Neural Compute Stick 1

2.1 NCSDK Installation

We FIRST need to have ncsdk installed. Yup, here, as described in Run NCS Applications on Raspberry Pi, we carry out the installation directly under folder ...../ncsdk/api/src.

1
2
3
➜  src git:(ncsdk2) ✗ make -j4
NCSDK FW successfully installed
➜  sudo make install

2.2 Test NCSDK Example Apps

1
2
3
4
➜  python hello_ncs.py
Hello NCS! Device opened normally.
Goodbye NCS! Device closed normally.
NCS device working.

2.3 mvnc Python Package

1
2
3
4
5
6
7
8
pi@raspberrypi:~ $ python
Python 3.7.3 (default, Apr  3 2019, 05:39:12) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mvnc
>>> mvnc.__file__
'/usr/local/lib/python3.7/dist-packages/mvnc/__init__.py'
>>>

3. Transitioning from Intel Movidius Neural Compute SDK to Intel OpenVINO

By following Intel’s official documentation Transitioning from Intel® Movidius™ Neural Compute SDK to Intel® Distribution of OpenVINO™ toolkit, we are transitioning to OpenVINO, which supports both Intel NCS 2 and the original NCS.

3.1 Install OpenVINO for Raspbian

For the installation details of OpenVINO, please refer to the following 2 documentations:

We now extract the MOST up-to-date l_openvino_toolkit_runtime_raspbian_p_2019.3.334.tgz under folder /opt/intel/openvino. Let’s take a brief look at this folder:

1
2
3
4
pi@raspberrypi:/opt/intel/openvino $ ls
bin  deployment_tools  documentation  inference_engine  install_dependencies  licensing  opencv  python
pi@raspberrypi:/opt/intel/openvino $ ls deployment_tools/
inference_engine

Clearly, by comparing with OpenVINO™ Toolkit - Deep Learning Deployment Toolkit repository, we know that the open source version of deployment_tools contains some more content than the trimmed version for Raspbian. We’ll use model-optimizer for sure. Therefore, we checked out dldt, and put two subfolders model-optimizer and tools under folder /opt/intel/openvino/deployment_tools.

1
2
pi@raspberrypi:/opt/intel/openvino/deployment_tools $ ls
inference_engine  open_model_zoo  tools

3.2 Build OpenVINO Samples

Before start building OpenVINO samples, please have OpenCV built from source and installed.

Note: Be sure to enable -DCMAKE_CXX_FLAGS=’-march=armv7-a’ while building dldt/samples.

Those samples are ALL for C++. For Python samples, you may have to figure out where to put openvino and where should those .so files put under. Please refer to Intel Forum Issue 810084 for the solutions to my problem. Anyway, the final step is to test as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
pi@raspberrypi:/opt/intel/openvino/inference_engine/samples/python_samples $ python hello_query_device/hello_query_device.py 
Available devices:
	Device: MYRIAD
	Metrics:
		AVAILABLE_DEVICES: 1.4.3-ma2480
		FULL_DEVICE_NAME: Intel Movidius Myriad X VPU
		SUPPORTED_METRICS: AVAILABLE_DEVICES, FULL_DEVICE_NAME, SUPPORTED_METRICS, SUPPORTED_CONFIG_KEYS, OPTIMIZATION_CAPABILITIES, RANGE_FOR_ASYNC_INFER_REQUESTS
		SUPPORTED_CONFIG_KEYS: VPU_HW_STAGES_OPTIMIZATION, VPU_LOG_LEVEL, VPU_PRINT_RECEIVE_TENSOR_TIME, VPU_NETWORK_CONFIG, VPU_COMPUTE_LAYOUT, VPU_CUSTOM_LAYERS, VPU_IGNORE_IR_STATISTIC, VPU_MYRIAD_FORCE_RESET, VPU_MYRIAD_PLATFORM, EXCLUSIVE_ASYNC_REQUESTS, LOG_LEVEL, PERF_COUNT, CONFIG_FILE, DEVICE_ID
		OPTIMIZATION_CAPABILITIES: FP16
		RANGE_FOR_ASYNC_INFER_REQUESTS: 3, 6, 1

	Default values for device configuration keys:
		VPU_HW_STAGES_OPTIMIZATION: UNSUPPORTED TYPE
		VPU_LOG_LEVEL: UNSUPPORTED TYPE
		VPU_PRINT_RECEIVE_TENSOR_TIME: UNSUPPORTED TYPE
		VPU_NETWORK_CONFIG: UNSUPPORTED TYPE
		VPU_COMPUTE_LAYOUT: UNSUPPORTED TYPE
		VPU_CUSTOM_LAYERS: UNSUPPORTED TYPE
		VPU_IGNORE_IR_STATISTIC: UNSUPPORTED TYPE
		VPU_MYRIAD_FORCE_RESET: UNSUPPORTED TYPE
		VPU_MYRIAD_PLATFORM: UNSUPPORTED TYPE
		EXCLUSIVE_ASYNC_REQUESTS: UNSUPPORTED TYPE
		LOG_LEVEL: UNSUPPORTED TYPE
		PERF_COUNT: UNSUPPORTED TYPE
		CONFIG_FILE: UNSUPPORTED TYPE
		DEVICE_ID: UNSUPPORTED TYPE

3.3 OpenVINO Test

Please refer to Device-specific Plugin Libraries for ALL possible device types.

3.3.1 Object Detection

We then download two model files as given in the blog Install OpenVINO™ toolkit for Raspbian* OS.

Afterwards, start running object_detection_sample_ssd as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
pi@raspberrypi:/opt/intel/openvino/inference_engine/samples/build $ ./armv7l/Release/object_detection_sample_ssd -m face-detection-adas-0001.xml -d MYRIAD -i me.jpg 
[ INFO ] InferenceEngine: 
	API version ............ 2.1
	Build .................. custom_releases/2019/R3_cb6cad9663aea3d282e0e8b3e0bf359df665d5d0
	Description ....... API
Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ]     me.jpg
[ INFO ] Loading Inference Engine
[ INFO ] Device info: 
	MYRIAD
	myriadPlugin version ......... 2.1
	Build ........... 30677
[ INFO ] Loading network files:
	face-detection-adas-0001.xml
	face-detection-adas-0001.bin
[ INFO ] Preparing input blobs
[ INFO ] Batch size is 1
[ INFO ] Preparing output blobs
[ INFO ] Loading model to the device
[ INFO ] Create infer request
[ WARNING ] Image is resized from (924, 1280) to (672, 384)
[ INFO ] Batch size is 1
[ INFO ] Start inference
[ INFO ] Processing output blobs
[0,1] element, prob = 0.870117    (421,202)-(745,608) batch id : 0 WILL BE PRINTED!
[1,1] element, prob = 0.0268555    (-55,556)-(234,1100) batch id : 0
[2,1] element, prob = 0.0219727    (20,925)-(239,1340) batch id : 0
[3,1] element, prob = 0.0200195    (-18,-144)-(179,581) batch id : 0
[4,1] element, prob = 0.0180664    (636,315)-(757,471) batch id : 0
[5,1] element, prob = 0.0170898    (-27,672)-(137,1217) batch id : 0
[6,1] element, prob = 0.0170898    (74,976)-(364,1322) batch id : 0
[7,1] element, prob = 0.0161133    (291,958)-(349,1122) batch id : 0
[8,1] element, prob = 0.0161133    (329,963)-(391,1126) batch id : 0
[9,1] element, prob = 0.0161133    (203,928)-(423,1346) batch id : 0
[10,1] element, prob = 0.015625    (694,1034)-(732,1140) batch id : 0
[11,1] element, prob = 0.015625    (832,1121)-(937,1295) batch id : 0
[12,1] element, prob = 0.015625    (219,39)-(388,627) batch id : 0
[13,1] element, prob = 0.015625    (76,785)-(357,1160) batch id : 0
[14,1] element, prob = 0.0146484    (355,616)-(475,778) batch id : 0
[15,1] element, prob = 0.0146484    (240,779)-(331,1048) batch id : 0
[16,1] element, prob = 0.0136719    (0,893)-(24,984) batch id : 0
[17,1] element, prob = 0.0136719    (658,306)-(752,405) batch id : 0
[18,1] element, prob = 0.0136719    (204,1198)-(267,1275) batch id : 0
[19,1] element, prob = 0.0136719    (322,611)-(430,794) batch id : 0
[20,1] element, prob = 0.0136719    (171,244)-(423,798) batch id : 0
[21,1] element, prob = 0.0136719    (252,798)-(539,1177) batch id : 0
[22,1] element, prob = 0.0136719    (133,875)-(301,1420) batch id : 0
[23,1] element, prob = 0.0136719    (443,202)-(802,1022) batch id : 0
[24,1] element, prob = 0.0126953    (718,466)-(757,540) batch id : 0
[25,1] element, prob = 0.0126953    (166,527)-(206,594) batch id : 0
[26,1] element, prob = 0.0126953    (694,511)-(731,590) batch id : 0
[27,1] element, prob = 0.0126953    (705,441)-(771,555) batch id : 0
[28,1] element, prob = 0.0126953    (272,962)-(322,1117) batch id : 0
[29,1] element, prob = 0.0126953    (268,486)-(376,688) batch id : 0
[30,1] element, prob = 0.0126953    (323,489)-(422,690) batch id : 0
[31,1] element, prob = 0.0126953    (277,607)-(384,796) batch id : 0
[32,1] element, prob = 0.0126953    (233,709)-(342,924) batch id : 0
[33,1] element, prob = 0.0126953    (256,672)-(406,957) batch id : 0
[34,1] element, prob = 0.0126953    (252,1043)-(431,1214) batch id : 0
[35,1] element, prob = 0.0117188    (695,466)-(733,539) batch id : 0
[36,1] element, prob = 0.0117188    (673,514)-(709,587) batch id : 0
[37,1] element, prob = 0.0117188    (370,1005)-(399,1078) batch id : 0
[38,1] element, prob = 0.0117188    (670,256)-(753,346) batch id : 0
[39,1] element, prob = 0.0117188    (151,500)-(221,610) batch id : 0
[40,1] element, prob = 0.0117188    (447,489)-(511,608) batch id : 0
[41,1] element, prob = 0.0117188    (126,625)-(200,719) batch id : 0
[42,1] element, prob = 0.0117188    (299,626)-(387,709) batch id : 0
[43,1] element, prob = 0.0117188    (164,673)-(251,747) batch id : 0
[44,1] element, prob = 0.0117188    (398,850)-(453,1008) batch id : 0
[45,1] element, prob = 0.0117188    (425,851)-(476,1008) batch id : 0
[46,1] element, prob = 0.0117188    (310,915)-(371,1077) batch id : 0
[47,1] element, prob = 0.0117188    (357,925)-(413,1073) batch id : 0
[48,1] element, prob = 0.0117188    (378,923)-(435,1074) batch id : 0
[49,1] element, prob = 0.0117188    (468,906)-(521,1070) batch id : 0
[50,1] element, prob = 0.0117188    (284,1016)-(357,1188) batch id : 0
[51,1] element, prob = 0.0117188    (203,-95)-(283,182) batch id : 0
[52,1] element, prob = 0.0117188    (269,378)-(374,595) batch id : 0
[53,1] element, prob = 0.0117188    (-22,575)-(71,831) batch id : 0
[54,1] element, prob = 0.0117188    (318,698)-(433,911) batch id : 0
[55,1] element, prob = 0.0117188    (221,716)-(395,1258) batch id : 0
[56,1] element, prob = 0.0117188    (859,771)-(987,1518) batch id : 0
[57,1] element, prob = 0.0107422    (145,466)-(185,527) batch id : 0
[58,1] element, prob = 0.0107422    (166,463)-(204,521) batch id : 0
[59,1] element, prob = 0.0107422    (185,477)-(224,537) batch id : 0
[60,1] element, prob = 0.0107422    (214,477)-(250,540) batch id : 0
[61,1] element, prob = 0.0107422    (458,478)-(488,536) batch id : 0
[62,1] element, prob = 0.0107422    (151,537)-(177,594) batch id : 0
[63,1] element, prob = 0.0107422    (460,522)-(488,590) batch id : 0
[64,1] element, prob = 0.0107422    (481,523)-(509,588) batch id : 0
[65,1] element, prob = 0.0107422    (716,512)-(755,592) batch id : 0
[66,1] element, prob = 0.0107422    (129,581)-(158,641) batch id : 0
[67,1] element, prob = 0.0107422    (150,579)-(179,635) batch id : 0
[68,1] element, prob = 0.0107422    (171,583)-(201,641) batch id : 0
[69,1] element, prob = 0.0107422    (457,564)-(491,658) batch id : 0
[70,1] element, prob = 0.0107422    (481,581)-(510,647) batch id : 0
[71,1] element, prob = 0.0107422    (652,565)-(689,650) batch id : 0
[72,1] element, prob = 0.0107422    (674,571)-(710,650) batch id : 0
[73,1] element, prob = 0.0107422    (694,571)-(734,649) batch id : 0
[74,1] element, prob = 0.0107422    (143,640)-(188,705) batch id : 0
[75,1] element, prob = 0.0107422    (191,743)-(224,805) batch id : 0
[76,1] element, prob = 0.0107422    (437,843)-(465,920) batch id : 0
[77,1] element, prob = 0.0107422    (-1,943)-(22,1037) batch id : 0
[78,1] element, prob = 0.0107422    (414,1007)-(443,1080) batch id : 0
[79,1] element, prob = 0.0107422    (896,1206)-(924,1279) batch id : 0
[80,1] element, prob = 0.0107422    (678,200)-(743,294) batch id : 0
[81,1] element, prob = 0.0107422    (70,374)-(190,512) batch id : 0
[82,1] element, prob = 0.0107422    (422,429)-(485,563) batch id : 0
[83,1] element, prob = 0.0107422    (106,477)-(213,629) batch id : 0
[84,1] element, prob = 0.0107422    (428,477)-(479,615) batch id : 0
[85,1] element, prob = 0.0107422    (648,498)-(727,605) batch id : 0
[86,1] element, prob = 0.0107422    (703,490)-(769,614) batch id : 0
[87,1] element, prob = 0.0107422    (16,531)-(150,712) batch id : 0
[88,1] element, prob = 0.0107422    (87,546)-(150,685) batch id : 0
[89,1] element, prob = 0.0107422    (127,552)-(197,653) batch id : 0
[90,1] element, prob = 0.0107422    (425,557)-(481,671) batch id : 0
[91,1] element, prob = 0.0107422    (437,509)-(516,700) batch id : 0
[92,1] element, prob = 0.0107422    (467,549)-(524,678) batch id : 0
[93,1] element, prob = 0.0107422    (633,551)-(704,664) batch id : 0
[94,1] element, prob = 0.0107422    (416,611)-(487,735) batch id : 0
[95,1] element, prob = 0.0107422    (466,602)-(530,734) batch id : 0
[96,1] element, prob = 0.0107422    (124,673)-(198,759) batch id : 0
[97,1] element, prob = 0.0107422    (156,705)-(260,849) batch id : 0
[98,1] element, prob = 0.0107422    (54,828)-(137,933) batch id : 0
[99,1] element, prob = 0.0107422    (451,812)-(498,960) batch id : 0
[100,1] element, prob = 0.0107422    (470,843)-(520,1004) batch id : 0
[101,1] element, prob = 0.0107422    (447,897)-(500,1066) batch id : 0
[102,1] element, prob = 0.0107422    (641,966)-(706,1122) batch id : 0
[103,1] element, prob = 0.0107422    (164,85)-(233,410) batch id : 0
[104,1] element, prob = 0.0107422    (188,497)-(307,688) batch id : 0
[105,1] element, prob = 0.0107422    (205,445)-(357,715) batch id : 0
[106,1] element, prob = 0.0107422    (230,602)-(341,791) batch id : 0
[107,1] element, prob = 0.0107422    (274,785)-(390,1021) batch id : 0
[108,1] element, prob = 0.0107422    (26,709)-(209,1239) batch id : 0
[109,1] element, prob = 0.0107422    (309,868)-(485,1410) batch id : 0
[110,1] element, prob = 0.0107422    (229,981)-(575,1306) batch id : 0
[111,1] element, prob = 0.0107422    (29,141)-(518,883) batch id : 0
[ INFO ] Image out_0.bmp created!
[ INFO ] Execution successful

[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
me.jpg out_0.bmp
Me Detected: Out_0

3.3.2 Image Classification

In this section, we are going to test out another OpenVINO example: Image Classification C++ Sample Async. After reading this blog, it wouldn’t be hard for us to notice the MOST important thing we’re missing here is the model file alexnet_fp32.xml. Let’s just keep it in mind for now.

And, let’s review a bit about our previous example: Object Detection – we downloaded face-detection-adas-0001 model from online and use it directly. So, questions:

  • Are we able to download alexnet_fp32.xml from online this time again?
  • Where can we download a whole bunch of open source models?

3.3.2.1 Open Model Zoo

It wouldn’t be hard for us to google out OpenVINO™ Toolkit - Open Model Zoo repository, under which model face-detection-adas-0001 is just sitting there. However, face-detection-adas-0001.xml and face-detection-adas-0001.bin are missing.

1
2
$ ls face-detection-adas-0001/
description  face-detection-adas-0001.prototxt  model.yml

Let’s checkout open_model_zoo and put it under folder /opt/intel/openvino/deployment_tools.

1
2
pi@raspberrypi:/opt/intel/openvino/deployment_tools $ ls
inference_engine  model_optimizer  open_model_zoo  tools

It’s seems that each folder under intel and public contains the detailed info of each model. For instance, file intel/face-detection-adas-0001/model.yml contains all the info about model face-detection-adas-0001.

However, what if ONLY caffe model is provided? In order to use those models on movidius, we need to generate two files, one .xml and one .bin, which are optimized for movidius, from some paticular caffe models. Intel OpenVINO toolkit issue 798441 provided a way to generate these two files.

3.3.2.2 Download Caffe Model Files

1
2
3
4
5
6
7
8
9
10
11
12
13
pi@raspberrypi:/opt/intel/openvino/deployment_tools $ cd ./open_model_zoo/tools/downloader/
pi@raspberrypi:/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader $ python downloader.py --name alexnet
################|| Downloading models ||################

========== Downloading /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/public/alexnet/alexnet.prototxt
... 100%, 3 KB, 9485 KB/s, 0 seconds passed

========== Downloading /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/public/alexnet/alexnet.caffemodel
... 100%, 238146 KB, 5054 KB/s, 47 seconds passed

################|| Post-processing ||################

========== Replacing text in /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/public/alexnet/alexnet.prototxt

Three files including a large model file alexnet.caffemodel has been downloaded.

1
2
3
4
5
pi@raspberrypi:/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader $ ll public/alexnet/
total 238160
-rw-r--r-- 1 pi pi 243862414 Jan  8 20:17 alexnet.caffemodel
-rw-r--r-- 1 pi pi      3628 Jan  8 20:17 alexnet.prototxt
-rw-r--r-- 1 pi pi      3629 Jan  8 16:32 alexnet.prototxt.orig

3.3.2.3 Model Optimization

We then need to optimize the downloaded caffe model and make it feedable to OpenVINO.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
pi@raspberrypi:/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader $ cd ../../../model_optimizer/
pi@raspberrypi:/opt/intel/openvino/deployment_tools/model_optimizer $ python mo.py --input_model ../open_model_zoo/tools/downloader/public/alexnet/alexnet.caffemodel
Model Optimizer arguments:
Common parameters:
	- Path to the Input Model: 	/opt/intel/openvino/deployment_tools/model_optimizer/../open_model_zoo/tools/downloader/public/alexnet/alexnet.caffemodel
	- Path for generated IR: 	/opt/intel/openvino/deployment_tools/model_optimizer/.
	- IR output name: 	alexnet
	- Log level: 	ERROR
	- Batch: 	Not specified, inherited from the model
	- Input layers: 	Not specified, inherited from the model
	- Output layers: 	Not specified, inherited from the model
	- Input shapes: 	Not specified, inherited from the model
	- Mean values: 	Not specified
	- Scale values: 	Not specified
	- Scale factor: 	Not specified
	- Precision of IR: 	FP32
	- Enable fusing: 	True
	- Enable grouped convolutions fusing: 	True
	- Move mean values to preprocess section: 	False
	- Reverse input channels: 	False
Caffe specific parameters:
	- Path to Python Caffe* parser generated from caffe.proto: 	mo/front/caffe/proto
	- Enable resnet optimization: 	True
	- Path to the Input prototxt: 	/opt/intel/openvino/deployment_tools/model_optimizer/../open_model_zoo/tools/downloader/public/alexnet/alexnet.prototxt
	- Path to CustomLayersMapping.xml: 	Default
	- Path to a mean file: 	Not specified
	- Offsets for a mean file: 	Not specified
Model Optimizer version: 	2019.3.0-408-gac8584cb7
Please expect that Model Optimizer conversion might be slow. You are currently using Python protobuf library implementation. 
However you can use the C++ protobuf implementation that is supplied with the OpenVINO toolkitor build protobuf library from sources. 
Navigate to "install_prerequisites" folder and run: python -m easy_install protobuf-3.5.1-py($your_python_version)-win-amd64.egg 
set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp


 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #80. 

[ SUCCESS ] Generated IR model.
[ SUCCESS ] XML file: /opt/intel/openvino/deployment_tools/model_optimizer/./alexnet.xml
[ SUCCESS ] BIN file: /opt/intel/openvino/deployment_tools/model_optimizer/./alexnet.bin
[ SUCCESS ] Total execution time: 676.37 seconds.

Note: You may meet the following ERRORs during model optimization.

1
2
3
4
[ ERROR ]  
Detected not satisfied dependencies:
	networkx: installed: 2.4, required: 2.4
	protobuf: installed: 3.11.2, required: 3.6.1

Clearly, for networkx, the ERROR message is a kind of ridiculous.
Anyway, if you meet the above 2 errors, please DOWNGRADE your packages as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
pi@raspberrypi:/opt/intel/openvino/deployment_tools/model_optimizer $ pip install protobuf==3.6.1 --user
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting protobuf==3.6.1
  Downloading https://files.pythonhosted.org/packages/77/78/a7f1ce761e2c738e209857175cd4f90a8562d1bde32868a8cd5290d58926/protobuf-3.6.1-py2.py3-none-any.whl (390kB)
     |████████████████████████████████| 399kB 1.4MB/s 
Requirement already satisfied: six>=1.9 in /home/pi/.local/lib/python3.7/site-packages (from protobuf==3.6.1) (1.13.0)
Requirement already satisfied: setuptools in /home/pi/.local/lib/python3.7/site-packages (from protobuf==3.6.1) (44.0.0)
Installing collected packages: protobuf
  Found existing installation: protobuf 3.11.2
    Uninstalling protobuf-3.11.2:
      Successfully uninstalled protobuf-3.11.2
Successfully installed protobuf-3.6.1
pi@raspberrypi:/opt/intel/openvino/deployment_tools/model_optimizer $ pip install networkx==2.3 --user
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting networkx==2.3
  Downloading https://files.pythonhosted.org/packages/85/08/f20aef11d4c343b557e5de6b9548761811eb16e438cee3d32b1c66c8566b/networkx-2.3.zip (1.7MB)
     |████████████████████████████████| 1.8MB 2.3MB/s 
Requirement already satisfied: decorator>=4.3.0 in /home/pi/.local/lib/python3.7/site-packages (from networkx==2.3) (4.4.1)
Building wheels for collected packages: networkx
  Building wheel for networkx (setup.py) ... done
  Created wheel for networkx: filename=networkx-2.3-py2.py3-none-any.whl size=1556408 sha256=9964dc8e3b41e97f0228afb2c1c6ca40a5dfc07dc1a2574b92717114f3d5e533
  Stored in directory: /home/pi/.cache/pip/wheels/de/63/64/3699be2a9d0ccdb37c7f16329acf3863fd76eda58c39c737af
Successfully built networkx
Installing collected packages: networkx
  Found existing installation: networkx 2.4
    Uninstalling networkx-2.4:
      Successfully uninstalled networkx-2.4
Successfully installed networkx-2.3

3.3.2.4 Run The Sample

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
pi@raspberrypi:/opt/intel/openvino/deployment_tools/inference_engine/samples/build $ ./armv7l/Release/classification_sample_async -i ./me.jpg -m /opt/intel/openvino/deployment_tools/model_optimizer/alexnet.xml -nt 5 -d HETERO:MYRIAD
[ INFO ] InferenceEngine: 
	API version ............ 2.1
	Build .................. custom_releases/2019/R3_cb6cad9663aea3d282e0e8b3e0bf359df665d5d0
	Description ....... API
[ INFO ] Parsing input parameters
[ INFO ] Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ]     ./me.jpg
[ INFO ] Creating Inference Engine
	HETERO
	heteroPlugin version ......... 2.1
	Build ........... heteroPlugin
	MYRIAD
	myriadPlugin version ......... 2.1
	Build ........... 30677

[ INFO ] Loading network files
[ INFO ] Preparing input blobs
[ WARNING ] Image is resized from (924, 1280) to (227, 227)
[ INFO ] Batch size is 1
[ INFO ] Loading model to the device
[ INFO ] Create infer request
[ INFO ] Start inference (10 asynchronous executions)
[ INFO ] Completed 1 async request execution
[ INFO ] Completed 2 async request execution
[ INFO ] Completed 3 async request execution
[ INFO ] Completed 4 async request execution
[ INFO ] Completed 5 async request execution
[ INFO ] Completed 6 async request execution
[ INFO ] Completed 7 async request execution
[ INFO ] Completed 8 async request execution
[ INFO ] Completed 9 async request execution
[ INFO ] Completed 10 async request execution
[ INFO ] Processing output blobs

Top 5 results:

Image ./me.jpg

classid probability
------- -----------
838     0.0751343  
978     0.0728149  
977     0.0722656  
975     0.0606079  
903     0.0533142  

[ INFO ] Execution successful

[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool

4. OpenCV DNN with OpenVINO’s Inference Engine

For the example openvino_fd_myriad.py given on Install OpenVINO™ toolkit for Raspbian* OS, the TOUGH thing is how to build [OpenCV])(https://opencv.org/) with OpenVINO’s Inference Engine. More or less, it’s a kind of complicated.

4.1 Intel64

On my laptop, of course, we are building OpenCV for architecture Intel64, with NVidia GPU + CUDA support, for DNN in OpenCV requires either CUDA or OpenCL.

The key file modified by me is opencv/cmake/OpenCVDetectInferenceEngine.cmake, as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
# The script detects Intel(R) Inference Engine installation
#
# Cache variables:
# INF_ENGINE_RELEASE - a number reflecting IE source interface (linked with OpenVINO release)
#
# Detect parameters:
# 1. Native cmake IE package:
#    - environment variable InferenceEngine_DIR is set to location of cmake module
# 2. Custom location:
#    - INF_ENGINE_INCLUDE_DIRS - headers search location
#    - INF_ENGINE_LIB_DIRS     - library search location
# 3. OpenVINO location:
#    - environment variable INTEL_OPENVINO_DIR is set to location of OpenVINO installation dir
#    - INF_ENGINE_PLATFORM - part of name of library directory representing its platform
#
# Result:
# INF_ENGINE_TARGET - set to name of imported library target representing InferenceEngine
#
# Modified by Pei Jia, 2020-01-22

function(add_custom_ie_build _inc _lib _lib_rel _lib_dbg _msg)
  if(NOT _inc OR NOT (_lib OR _lib_rel OR _lib_dbg))
    return()
  endif()
  add_library(inference_engine UNKNOWN IMPORTED)
  set_target_properties(inference_engine PROPERTIES
    IMPORTED_LOCATION "${_lib}"
    IMPORTED_IMPLIB_RELEASE "${_lib_rel}"
    IMPORTED_IMPLIB_DEBUG "${_lib_dbg}"
    INTERFACE_INCLUDE_DIRECTORIES "${_inc}"
  )

  find_library(ie_builder_custom_lib "inference_engine_nn_builder" PATHS "${INF_ENGINE_LIB_DIRS}" NO_DEFAULT_PATH)
  if(EXISTS "${ie_builder_custom_lib}")
    add_library(inference_engine_nn_builder UNKNOWN IMPORTED)
    set_target_properties(inference_engine_nn_builder PROPERTIES
      IMPORTED_LOCATION "${ie_builder_custom_lib}"
    )
  endif()

  if(NOT INF_ENGINE_RELEASE VERSION_GREATER "2018050000")
    find_library(INF_ENGINE_OMP_LIBRARY iomp5 PATHS "${INF_ENGINE_OMP_DIR}" NO_DEFAULT_PATH)
    if(NOT INF_ENGINE_OMP_LIBRARY)
      message(WARNING "OpenMP for IE have not been found. Set INF_ENGINE_OMP_DIR variable if you experience build errors.")
    else()
      set_target_properties(inference_engine PROPERTIES IMPORTED_LINK_INTERFACE_LIBRARIES "${INF_ENGINE_OMP_LIBRARY}")
    endif()
  endif()
  set(INF_ENGINE_VERSION "Unknown" CACHE STRING "")
  set(INF_ENGINE_TARGET inference_engine)
  if(TARGET inference_engine_nn_builder)
    list(APPEND INF_ENGINE_TARGET inference_engine_nn_builder)
    set(_msg "${_msg}, with IE NN Builder API")
  endif()
  set(INF_ENGINE_TARGET "${INF_ENGINE_TARGET}" PARENT_SCOPE)
  message(STATUS "Detected InferenceEngine: ${_msg}")
endfunction()

# ======================

find_package(InferenceEngine QUIET)
if(InferenceEngine_FOUND)
  set(INF_ENGINE_TARGET ${InferenceEngine_LIBRARIES})
  set(INF_ENGINE_VERSION "${InferenceEngine_VERSION}" CACHE STRING "")
  message(STATUS "Detected InferenceEngine: cmake package (${InferenceEngine_VERSION})")
endif()

set(INF_ENGINE_LIB_DIRS "/opt/intel/openvino/deployment_tools/inference_engine/include")
set(INF_ENGINE_INCLUDE_DIRS "/opt/intel/openvino/deployment_tools/inference_engine/lib/intel64")
message(${INF_ENGINE_TARGET})
message(${INF_ENGINE_LIB_DIRS})
message(${INF_ENGINE_INCLUDE_DIRS})
#if(NOT INF_ENGINE_TARGET AND INF_ENGINE_LIB_DIRS AND INF_ENGINE_INCLUDE_DIRS)
if(INF_ENGINE_LIB_DIRS AND INF_ENGINE_INCLUDE_DIRS)
  find_path(ie_custom_inc "inference_engine.hpp" PATHS "${INF_ENGINE_INCLUDE_DIRS}" NO_DEFAULT_PATH)
  find_library(ie_custom_lib "inference_engine" PATHS "${INF_ENGINE_LIB_DIRS}" NO_DEFAULT_PATH)
  find_library(ie_custom_lib_rel "inference_engine" PATHS "${INF_ENGINE_LIB_DIRS}/Release" NO_DEFAULT_PATH)
  find_library(ie_custom_lib_dbg "inference_engine" PATHS "${INF_ENGINE_LIB_DIRS}/Debug" NO_DEFAULT_PATH)
  add_custom_ie_build("${ie_custom_inc}" "${ie_custom_lib}" "${ie_custom_lib_rel}" "${ie_custom_lib_dbg}" "INF_ENGINE_{INCLUDE,LIB}_DIRS")
message("okokok")
endif()

set(_loc "$ENV{INTEL_OPENVINO_DIR}")
if(NOT _loc AND DEFINED ENV{INTEL_CVSDK_DIR})
  set(_loc "$ENV{INTEL_CVSDK_DIR}")  # OpenVINO 2018.x
endif()
message(${_loc})
message(${INF_ENGINE_TARGET})
if(NOT INF_ENGINE_TARGET AND _loc)
  if(NOT INF_ENGINE_RELEASE VERSION_GREATER "2018050000")
    set(INF_ENGINE_PLATFORM_DEFAULT "ubuntu_18.04")
  else()
    set(INF_ENGINE_PLATFORM_DEFAULT "")
  endif()
message("INF_ENGINE_PLATFORM_DEFAULT is empty")
  set(INF_ENGINE_PLATFORM "${INF_ENGINE_PLATFORM_DEFAULT}" CACHE STRING "InferenceEngine platform (library dir)")
message(${INF_ENGINE_PLATFORM})
  find_path(ie_custom_env_inc "inference_engine.hpp" PATHS "${_loc}/deployment_tools/inference_engine/include" NO_DEFAULT_PATH)
  find_library(ie_custom_env_lib "inference_engine" PATHS "${_loc}/deployment_tools/inference_engine/lib/${INF_ENGINE_PLATFORM}/intel64" NO_DEFAULT_PATH)
  find_library(ie_custom_env_lib_rel "inference_engine" PATHS "${_loc}/deployment_tools/inference_engine/lib/intel64/Release" NO_DEFAULT_PATH)
  find_library(ie_custom_env_lib_dbg "inference_engine" PATHS "${_loc}/deployment_tools/inference_engine/lib/intel64/Debug" NO_DEFAULT_PATH)
  add_custom_ie_build("${ie_custom_env_inc}" "${ie_custom_env_lib}" "${ie_custom_env_lib_rel}" "${ie_custom_env_lib_dbg}" "OpenVINO (${_loc})")
endif()

# Add more features to the target

if(INF_ENGINE_TARGET)
  if(NOT INF_ENGINE_RELEASE)
    message(WARNING "InferenceEngine version have not been set, 2019R3 will be used by default. Set INF_ENGINE_RELEASE variable if you experience build errors.")
  endif()
  set(INF_ENGINE_RELEASE "2019030000" CACHE STRING "Force IE version, should be in form YYYYAABBCC (e.g. 2018R2.0.2 -> 2018020002)")
  set_target_properties(${INF_ENGINE_TARGET} PROPERTIES
    INTERFACE_COMPILE_DEFINITIONS "HAVE_INF_ENGINE=1;INF_ENGINE_RELEASE=${INF_ENGINE_RELEASE}"
  )
endif()

if(WITH_NGRAPH)
  find_package(ngraph QUIET)
  if(ngraph_FOUND)
    ocv_assert(TARGET ngraph::ngraph)
    if(INF_ENGINE_RELEASE VERSION_LESS "2019039999")
      message(WARNING "nGraph is not tested with current InferenceEngine version: INF_ENGINE_RELEASE=${INF_ENGINE_RELEASE}")
    endif()
    message(STATUS "Detected ngraph: cmake package (${ngraph_VERSION})")
    set(HAVE_NGRAPH ON)
  endif()
endif()

And, my test result of openvino_fd_myriad.py shows the performance of adopted model face-detection-adas-0001 is NOT as good as expected.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import cv2 as cv
# Load the model.
net = cv.dnn.readNet('face-detection-adas-0001.xml',
                     'face-detection-adas-0001.bin')
# Specify target device.
net.setPreferableTarget(cv.dnn.DNN_TARGET_MYRIAD)
# Read an image.
frame = cv.imread('./parents.jpg')
if frame is None:
    raise Exception('Image not found!')
# Prepare input blob and perform an inference.
blob = cv.dnn.blobFromImage(frame, size=(960, 1280), ddepth=cv.CV_8U)
net.setInput(blob)
out = net.forward()
count = 0
# Draw detected faces on the frame.
for detection in out.reshape(-1, 7):
    confidence = float(detection[2])
    xmin = int(detection[3] * frame.shape[1])
    ymin = int(detection[4] * frame.shape[0])
    xmax = int(detection[5] * frame.shape[1])
    ymax = int(detection[6] * frame.shape[0])
    if confidence > 0.6:
        cv.rectangle(frame, (xmin, ymin), (xmax, ymax), color=(0, 255, 0))
        count += 1

print(count)
# Save the frame to an image file.
cv.imwrite('out.png', frame)
Parents At First Starbucks
Parents At First Starbucks
Detected Faces

4.2 armv7l

Modify opencv/cmake/OpenCVDetectInferenceEngine.cmake correspondingly as:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
# The script detects Intel(R) Inference Engine installation
#
# Cache variables:
# INF_ENGINE_RELEASE - a number reflecting IE source interface (linked with OpenVINO release)
#
# Detect parameters:
# 1. Native cmake IE package:
#    - environment variable InferenceEngine_DIR is set to location of cmake module
# 2. Custom location:
#    - INF_ENGINE_INCLUDE_DIRS - headers search location
#    - INF_ENGINE_LIB_DIRS     - library search location
# 3. OpenVINO location:
#    - environment variable INTEL_OPENVINO_DIR is set to location of OpenVINO installation dir
#    - INF_ENGINE_PLATFORM - part of name of library directory representing its platform
#
# Result:
# INF_ENGINE_TARGET - set to name of imported library target representing InferenceEngine
#

function(add_custom_ie_build _inc _lib _lib_rel _lib_dbg _msg)
  if(NOT _inc OR NOT (_lib OR _lib_rel OR _lib_dbg))
    return()
  endif()
  add_library(inference_engine UNKNOWN IMPORTED)
  set_target_properties(inference_engine PROPERTIES
    IMPORTED_LOCATION "${_lib}"
    IMPORTED_IMPLIB_RELEASE "${_lib_rel}"
    IMPORTED_IMPLIB_DEBUG "${_lib_dbg}"
    INTERFACE_INCLUDE_DIRECTORIES "${_inc}"
  )

  find_library(ie_builder_custom_lib "inference_engine_nn_builder" PATHS "${INF_ENGINE_LIB_DIRS}" NO_DEFAULT_PATH)
  if(EXISTS "${ie_builder_custom_lib}")
    add_library(inference_engine_nn_builder UNKNOWN IMPORTED)
    set_target_properties(inference_engine_nn_builder PROPERTIES
      IMPORTED_LOCATION "${ie_builder_custom_lib}"
    )
  endif()

  if(NOT INF_ENGINE_RELEASE VERSION_GREATER "2018050000")
    find_library(INF_ENGINE_OMP_LIBRARY iomp5 PATHS "${INF_ENGINE_OMP_DIR}" NO_DEFAULT_PATH)
    if(NOT INF_ENGINE_OMP_LIBRARY)
      message(WARNING "OpenMP for IE have not been found. Set INF_ENGINE_OMP_DIR variable if you experience build errors.")
    else()
      set_target_properties(inference_engine PROPERTIES IMPORTED_LINK_INTERFACE_LIBRARIES "${INF_ENGINE_OMP_LIBRARY}")
    endif()
  endif()
  set(INF_ENGINE_VERSION "Unknown" CACHE STRING "")
  set(INF_ENGINE_TARGET inference_engine)
  if(TARGET inference_engine_nn_builder)
    list(APPEND INF_ENGINE_TARGET inference_engine_nn_builder)
    set(_msg "${_msg}, with IE NN Builder API")
  endif()
  set(INF_ENGINE_TARGET "${INF_ENGINE_TARGET}" PARENT_SCOPE)
  message(STATUS "Detected InferenceEngine: ${_msg}")
endfunction()

# ======================

find_package(InferenceEngine QUIET)
if(InferenceEngine_FOUND)
  set(INF_ENGINE_TARGET ${InferenceEngine_LIBRARIES})
  set(INF_ENGINE_VERSION "${InferenceEngine_VERSION}" CACHE STRING "")
  message(STATUS "Detected InferenceEngine: cmake package (${InferenceEngine_VERSION})")
endif()

set(INF_ENGINE_LIB_DIRS "/opt/intel/openvino/deployment_tools/inference_engine/include")
set(INF_ENGINE_INCLUDE_DIRS "/opt/intel/openvino/deployment_tools/inference_engine/lib/armv7l")
#message(${INF_ENGINE_TARGET})
message(${INF_ENGINE_LIB_DIRS})
message(${INF_ENGINE_INCLUDE_DIRS})
#if(NOT INF_ENGINE_TARGET AND INF_ENGINE_LIB_DIRS AND INF_ENGINE_INCLUDE_DIRS)
if(INF_ENGINE_LIB_DIRS AND INF_ENGINE_INCLUDE_DIRS)
  find_path(ie_custom_inc "inference_engine.hpp" PATHS "${INF_ENGINE_INCLUDE_DIRS}" NO_DEFAULT_PATH)
  find_library(ie_custom_lib "inference_engine" PATHS "${INF_ENGINE_LIB_DIRS}" NO_DEFAULT_PATH)
  find_library(ie_custom_lib_rel "inference_engine" PATHS "${INF_ENGINE_LIB_DIRS}/Release" NO_DEFAULT_PATH)
  find_library(ie_custom_lib_dbg "inference_engine" PATHS "${INF_ENGINE_LIB_DIRS}/Debug" NO_DEFAULT_PATH)
  add_custom_ie_build("${ie_custom_inc}" "${ie_custom_lib}" "${ie_custom_lib_rel}" "${ie_custom_lib_dbg}" "INF_ENGINE_{INCLUDE,LIB}_DIRS")
message("okokok")
endif()

set(_loc "$ENV{INTEL_OPENVINO_DIR}")
if(NOT _loc AND DEFINED ENV{INTEL_CVSDK_DIR})
  set(_loc "$ENV{INTEL_CVSDK_DIR}")  # OpenVINO 2018.x
endif()
message(${_loc})
#message(${INF_ENGINE_TARGET})
if(NOT INF_ENGINE_TARGET AND _loc)
  if(NOT INF_ENGINE_RELEASE VERSION_GREATER "2018050000")
    set(INF_ENGINE_PLATFORM_DEFAULT "rpi4")
  else()
    set(INF_ENGINE_PLATFORM_DEFAULT "")
  endif()
message("INF_ENGINE_PLATFORM_DEFAULT is empty")
  set(INF_ENGINE_PLATFORM "${INF_ENGINE_PLATFORM_DEFAULT}" CACHE STRING "InferenceEngine platform (library dir)")
message(${INF_ENGINE_PLATFORM})
  find_path(ie_custom_env_inc "inference_engine.hpp" PATHS "${_loc}/deployment_tools/inference_engine/include" NO_DEFAULT_PATH)
  find_library(ie_custom_env_lib "inference_engine" PATHS "${_loc}/deployment_tools/inference_engine/lib/${INF_ENGINE_PLATFORM}/armv7l" NO_DEFAULT_PATH)
  find_library(ie_custom_env_lib_rel "inference_engine" PATHS "${_loc}/deployment_tools/inference_engine/lib/armv7l/Release" NO_DEFAULT_PATH)
  find_library(ie_custom_env_lib_dbg "inference_engine" PATHS "${_loc}/deployment_tools/inference_engine/lib/armv7l/Debug" NO_DEFAULT_PATH)
  add_custom_ie_build("${ie_custom_env_inc}" "${ie_custom_env_lib}" "${ie_custom_env_lib_rel}" "${ie_custom_env_lib_dbg}" "OpenVINO (${_loc})")
endif()

# Add more features to the target

if(INF_ENGINE_TARGET)
  if(NOT INF_ENGINE_RELEASE)
    message(WARNING "InferenceEngine version have not been set, 2019R3 will be used by default. Set INF_ENGINE_RELEASE variable if you experience build errors.")
  endif()
  set(INF_ENGINE_RELEASE "2019030000" CACHE STRING "Force IE version, should be in form YYYYAABBCC (e.g. 2018R2.0.2 -> 2018020002)")
  set_target_properties(${INF_ENGINE_TARGET} PROPERTIES
    INTERFACE_COMPILE_DEFINITIONS "HAVE_INF_ENGINE=1;INF_ENGINE_RELEASE=${INF_ENGINE_RELEASE}"
  )
endif()

if(WITH_NGRAPH)
  find_package(ngraph QUIET)
  if(ngraph_FOUND)
    ocv_assert(TARGET ngraph::ngraph)
    if(INF_ENGINE_RELEASE VERSION_LESS "2019039999")
      message(WARNING "nGraph is not tested with current InferenceEngine version: INF_ENGINE_RELEASE=${INF_ENGINE_RELEASE}")
    endif()
    message(STATUS "Detected ngraph: cmake package (${ngraph_VERSION})")
    set(HAVE_NGRAPH ON)
  endif()
endif()

OpenCV dnn net.forward() fails on armv7l

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
pi@raspberrypi:/opt/intel/openvino/deployment_tools/inference_engine/samples/build $ python openvino_fd_myriad.py 
[[[[238 238 238 ... 240 240 240]
   [238 238 238 ... 240 240 240]
   [239 239 239 ... 241 241 241]
   ...
   [130 131 132 ...  78  70  66]
   [130 132 132 ...  73  71  74]
   [110 120 126 ...  66  71  80]]

  [[160 160 160 ... 153 153 153]
   [161 161 161 ... 153 153 153]
   [161 161 161 ... 154 154 154]
   ...
   [135 136 137 ... 115 107 103]
   [136 137 137 ... 107 107 111]
   [119 127 132 ...  99 106 116]]

  [[ 91  91  91 ...  86  86  86]
   [ 91  91  91 ...  86  86  86]
   [ 92  92  92 ...  87  87  87]
   ...
   [134 135 137 ... 137 129 125]
   [138 139 139 ... 126 126 130]
   [123 130 135 ... 116 122 132]]]]
net set
double free or corruption (out)
Aborted

5. My Built Raspbian ISO With OpenCV4 + OpenVINO

Finally, you are welcome to try my built image rpi4-raspbian-opencv4-openvino-ncsdk.img, which is about 10G and composed of:

Everything has ALREADY been updated and built successfully.

Posted on | Edited on | In ,

The autumn in both Vancouver and Seattle is gorgeous…

Overview Seattle on Space Needle Around Space Needle
Overview Seattle on Space Needle Around Space Needle
Space Needle So Heavy
Space Needle So Heavy
Ferris Wheel Pier 55
Ferris Wheel Pier 55
Vancouver Maple UBC Poisonous Mushroom
Vancouver Maple UBC Poisonous Mushroom

Alright, let’s rapidly test Detectron2.

Installation is detailedly summarized in INSTALL.md.

We can simply follow GETTING_STARTED.md for some simple demonstrations. Make sure you’ve downloaded the demo pictures from Detectron1 demo and save under Detectron2’s folder demo.

1
2
3
4
5
6
7
8
longervision-GT72-6QE% python demo/demo.py --config-file configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml \
  --input ./demo/17790319373_bd19b24cfc_k.jpg \
  --output ./result.jpg \
  --opts MODEL.WEIGHTS detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl
[10/13 17:47:01 detectron2]: Arguments: Namespace(confidence_threshold=0.5, config_file='configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml', input=['./demo/17790319373_bd19b24cfc_k.jpg'], opts=['MODEL.WEIGHTS', 'detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl'], output='./result.jpg', video_input=None, webcam=False)
WARNING [10/13 17:47:01 d2.config.compat]: Config 'configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml' has no VERSION. Assuming it to be compatible with latest v2.
  0%|                                                                                                                                                                                        | 0/1 [00:00<?, ?it/s][10/13 17:47:05 detectron2]: ./demo/17790319373_bd19b24cfc_k.jpg: detected 13 instances in 0.29s
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  2.59it/s]

Let’s take a look at the result:

17790319373_bd19b24cfc_k

And of course, my pictures taken in Seattle and Vancouver:

Overview Seattle on Space Needle Around Space Needle
Overview Seattle on Space Needle Around Space Needle
Space Needle So Heavy
Space Needle So Heavy
Ferris Wheel Pier 55
Ferris Wheel Pier 55
Vancouver Maple UBC Poisonous Mushroom

Oooooops… it seems there are still quite a lot of ERROR detections…
Anyway… tomorrow, 热干面. Google translator: Hot Noodles with Sesame Paste…

Posted on | In ,

Flying back to China soon. Before leaving, here comes a simple blog testing Tensorflow 2.0. In this blog, I strictly follow Amita Kapoor and Ajit Jaokar’s free book Getting Started with TensorFlow 2.0.

For simplicity, let’s try it out directly:

1. Tensorflow 2.0

2. Tensorflow Dataset

Posted on | Edited on | In

Today, we’re going to build our own toolchain using crosstool-NG. There are so many cases that we want to build a particular toolchain for a specific embedded system development board. One MOST important reason for that is probably because the specific development board has limited resource to build upon, therefore, the time-consumption to build some software on the development board is VERY inefficient. In the other way around, cross compiling on the host PC is MORE efficient with respective to the time-consumption.

In this blog, for simplicity, we take Raspberry Pi 3B as our demo development board, for which we are building the cross compiler. Some references can be found at:

1. Installation

How to install crosstool-NG are thoroughly summarized at its official website. In my case, I had it installed under folder /opt/compilers/crosstool-ng. Let’s take a look:

1
2
3
4
5
6
longervision-GT72-6QE% pwd
/opt/compilers/crosstool-ng
longervision-GT72-6QE% ls
bin  libexec  share
longervision-GT72-6QE% ls bin
ct-ng

And make sure /opt/compilers/crosstool-ng/bin is under environment variable PATH.

2. Configuration

Under any directory that you want to save your .config file, we can configure our target cross compiler.

1
longervision-GT72-6QE% ct-ng menuconfig

Crosstool-NG

According to elinux: RPi Linaro GCC Compilation, we need to do the following selections:

  • Paths and misc options:

    • Try features marked as EXPERIMENTAL: ticked
    • Prefix directory: input the full path where you want to save the built toolchains
    • Number of parallel jobs: 4. Sorry that I’ve NO idea if this is the number of cores on Raspberry Pi 3B, but Raspberry Pi 3B does have 4 cores.

  • Target options:

    • Target Architecture: arm
    • Default instruction set mode: arm
    • Use EABI: ticked
    • append hf to the tuple (EXPERIMENTAL): ticked
    • Endianness: Little endian
    • Bitness: 32-bit
    • Emit assembly for CPU: cortex-a53
    • Use specific FPU: vfp
    • Floating point: hardware (FPU)

  • Toolchain options:

    • Tuple’s vendor string: rpi

  • Operating System

    1
    2
    ➜  ~ uname -a
        Linux raspberrypi 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux

  • Binary utilities:

    • Binary format: ELF
    • Version of binutils: 2.31.1
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    ➜  ~ apt show binutils
        Package: binutils
        Version: 2.31.1-16+rpi1
        Priority: optional
        Section: devel
        Maintainer: Matthias Klose <doko@debian.org>
        Installed-Size: 95.2 kB
        Provides: binutils-gold, elf-binutils
        Depends: binutils-common (= 2.31.1-16+rpi1), libbinutils (= 2.31.1-16+rpi1), binutils-arm-linux-gnueabihf (= 2.31.1-16+rpi1)
        Suggests: binutils-doc (>= 2.31.1-16+rpi1)
        Conflicts: binutils-multiarch (<< 2.27-8), modutils (<< 2.4.19-1)
        Homepage: https://www.gnu.org/software/binutils/
        Download-Size: 56.9 kB
        APT-Manual-Installed: no
        APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
        Description: GNU assembler, linker and binary utilities
        The programs in this package are used to assemble, link and manipulate
        binary and object files.  They may be used in conjunction with a compiler
        and various libraries to build programs.

  • C-library:

    • C library: glibc
    • Version of glibc: 2.28
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    ➜  ~ apt show libc-bin
        Package: libc-bin
        Version: 2.28-10+rpi1
        Priority: required
        Essential: yes
        Section: libs
        Source: glibc
        Maintainer: GNU Libc Maintainers <debian-glibc@lists.debian.org>
        Installed-Size: 3,015 kB
        Depends: libc6 (>> 2.28), libc6 (<< 2.29)
        Recommends: manpages
        Homepage: https://www.gnu.org/software/libc/libc.html
        Download-Size: 657 kB
        APT-Manual-Installed: yes
        APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
        Description: GNU C Library: Binaries
        This package contains utility programs related to the GNU C Library.
        .
        * catchsegv: catch segmentation faults in programs
        * getconf: query system configuration variables
        * getent: get entries from administrative databases
        * iconv, iconvconfig: convert between character encodings
        * ldd, ldconfig: print/configure shared library dependencies
        * locale, localedef: show/generate locale definitions
        * tzselect, zdump, zic: select/dump/compile time zones

  • C compiler:

    • Show gcc versions from: GNU
    • Version of gcc: 8.3.0
    • gcc extra config: --with-float=hard
    • Link libstdc++ statically into the gcc binary: tick
    • C++: tick
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    ➜  ~ apt show gcc
    Package: gcc
    Version: 4:8.3.0-1+rpi2
    Priority: optional
    Section: devel
    Source: gcc-defaults (1.181+rpi2)
    Maintainer: Debian GCC Maintainers <debian-gcc@lists.debian.org>
    Installed-Size: 46.1 kB
    Provides: c-compiler, gcc-arm-linux-gnueabihf (= 4:8.3.0-1+rpi2)
    Depends: cpp (= 4:8.3.0-1+rpi2), gcc-8 (>= 8.3.0-1~)
    Recommends: libc6-dev | libc-dev
    Suggests: gcc-multilib, make, manpages-dev, autoconf, automake, libtool, flex, bison, gdb, gcc-doc
    Conflicts: gcc-doc (<< 1:2.95.3)
    Download-Size: 5,200 B
    APT-Manual-Installed: no
    APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
    Description: GNU C compiler
    This is the GNU C compiler, a fairly portable optimizing compiler for C.
    .
    This is a dependency package providing the default GNU C compiler.

After we save the configuration to file .config, we Exit the crosstool-NG configuration dialog.

3. Build

Please remember to:

  • unset LD_LIBRARY_PATH before building. Otherwise, you’ll meet some ERROR messages.
  • mkdir ~/src before building. Otherwise, whenever you tried to rerun ct-ng build, you’ll have to download ALL required packages from scratch.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
longervision-GT72-6QE% ct-ng build
[INFO ]  Performing some trivial sanity checks
[WARN ]  Number of open files 1024 may not be sufficient to build the toolchain; increasing to 2048
[INFO ]  Build started 20190906.014128
[INFO ]  Building environment variables
[EXTRA]  Preparing working directories
[EXTRA]  Installing user-supplied crosstool-NG configuration
[EXTRA]  =================================================================
[EXTRA]  Dumping internal crosstool-NG configuration
[EXTRA]    Building a toolchain for:
[EXTRA]      build  = x86_64-pc-linux-gnu
[EXTRA]      host   = x86_64-pc-linux-gnu
[EXTRA]      target = arm-rpi-linux-gnueabihf
[EXTRA]  Dumping internal crosstool-NG configuration: done in 0.07s (at 00:01)
[INFO ]  =================================================================
[INFO ]  Retrieving needed toolchain components' tarballs
[EXTRA]    Retrieving 'ncurses-6.1'
[EXTRA]    Verifying SHA512 checksum for 'ncurses-6.1.tar.gz'
[EXTRA]    Saving 'ncurses-6.1.tar.gz' to local storage
[EXTRA]    Retrieving 'libiconv-1.15'
[EXTRA]    Verifying SHA512 checksum for 'libiconv-1.15.tar.gz'
[EXTRA]    Saving 'libiconv-1.15.tar.gz' to local storage
[EXTRA]    Retrieving 'gettext-0.19.8.1'
[EXTRA]    Verifying SHA512 checksum for 'gettext-0.19.8.1.tar.xz'
[EXTRA]    Saving 'gettext-0.19.8.1.tar.xz' to local storage
[EXTRA]    Retrieving 'binutils-2.31.1'
[EXTRA]    Verifying SHA512 checksum for 'binutils-2.31.1.tar.xz'
[EXTRA]    Saving 'binutils-2.31.1.tar.xz' to local storage
[EXTRA]    Retrieving 'gcc-8.3.0'
[EXTRA]    Verifying SHA512 checksum for 'gcc-8.3.0.tar.xz'
[EXTRA]    Saving 'gcc-8.3.0.tar.xz' to local storage
[EXTRA]    Retrieving 'glibc-2.28'
[EXTRA]    Verifying SHA512 checksum for 'glibc-2.28.tar.xz'
[EXTRA]    Saving 'glibc-2.28.tar.xz' to local storage
[INFO ]  Retrieving needed toolchain components' tarballs: done in 130.35s (at 02:12)
[INFO ]  =================================================================
[INFO ]  Extracting and patching toolchain components
[EXTRA]    Extracting linux-4.20.8
[EXTRA]    Patching linux-4.20.8
[EXTRA]    Extracting zlib-1.2.11
[EXTRA]    Patching zlib-1.2.11
[EXTRA]    Extracting gmp-6.1.2
[EXTRA]    Patching gmp-6.1.2
[EXTRA]    Extracting mpfr-4.0.2
[EXTRA]    Patching mpfr-4.0.2
[EXTRA]    Extracting isl-0.20
[EXTRA]    Patching isl-0.20
[EXTRA]    Extracting mpc-1.1.0
[EXTRA]    Patching mpc-1.1.0
[EXTRA]    Extracting ncurses-6.1
[EXTRA]    Patching ncurses-6.1
[EXTRA]    Extracting libiconv-1.15
[EXTRA]    Patching libiconv-1.15
[EXTRA]    Extracting gettext-0.19.8.1
[EXTRA]    Patching gettext-0.19.8.1
[EXTRA]    Extracting binutils-2.31.1
[EXTRA]    Patching binutils-2.31.1
[EXTRA]    Extracting gcc-8.3.0
[EXTRA]    Patching gcc-8.3.0
[EXTRA]    Extracting glibc-2.28
[EXTRA]    Patching glibc-2.28
[INFO ]  Extracting and patching toolchain components: done in 33.39s (at 02:45)
[INFO ]  =================================================================
[INFO ]  Installing ncurses for build
[EXTRA]    Configuring ncurses
[EXTRA]    Building ncurses
[EXTRA]    Installing ncurses
[INFO ]  Installing ncurses for build: done in 14.62s (at 03:00)
[INFO ]  =================================================================
[INFO ]  Installing zlib for host
[EXTRA]    Configuring zlib
[EXTRA]    Building zlib
[EXTRA]    Installing zlib
[INFO ]  Installing zlib for host: done in 1.12s (at 03:01)
[INFO ]  =================================================================
[INFO ]  Installing GMP for host
[EXTRA]    Configuring GMP
[EXTRA]    Building GMP
[EXTRA]    Installing GMP
[INFO ]  Installing GMP for host: done in 30.12s (at 03:31)
[INFO ]  =================================================================
[INFO ]  Installing MPFR for host
[EXTRA]    Configuring MPFR
[EXTRA]    Building MPFR
[EXTRA]    Installing MPFR
[INFO ]  Installing MPFR for host: done in 17.35s (at 03:48)
[INFO ]  =================================================================
[INFO ]  Installing ISL for host
[EXTRA]    Configuring ISL
[EXTRA]    Building ISL
[EXTRA]    Installing ISL
[INFO ]  Installing ISL for host: done in 17.73s (at 04:06)
[INFO ]  =================================================================
[INFO ]  Installing MPC for host
[EXTRA]    Configuring MPC
[EXTRA]    Building MPC
[EXTRA]    Installing MPC
[INFO ]  Installing MPC for host: done in 5.47s (at 04:12)
[INFO ]  =================================================================
[INFO ]  Installing ncurses for host
[EXTRA]    Configuring ncurses
[EXTRA]    Building ncurses
[EXTRA]    Installing ncurses
[INFO ]  Installing ncurses for host: done in 13.64s (at 04:25)
[INFO ]  =================================================================
[INFO ]  Installing libiconv for host
[EXTRA]    Skipping (included in GNU C library)
[INFO ]  Installing libiconv for host: done in 0.01s (at 04:25)
[INFO ]  =================================================================
[INFO ]  Installing gettext for host
[EXTRA]    Skipping (included in GNU C library)
[INFO ]  Installing gettext for host: done in 0.01s (at 04:25)
[INFO ]  =================================================================
[INFO ]  Installing binutils for host
[EXTRA]    Configuring binutils
[EXTRA]    Building binutils
[EXTRA]    Installing binutils
[INFO ]  Installing binutils for host: done in 46.89s (at 05:12)
[INFO ]  =================================================================
[INFO ]  Installing pass-1 core C gcc compiler
[EXTRA]    Configuring core C gcc compiler
[EXTRA]    Building gcc
[EXTRA]    Installing gcc
[EXTRA]    Housekeeping for core gcc compiler
[EXTRA]       '' --> lib (gcc)   lib (os)
[INFO ]  Installing pass-1 core C gcc compiler: done in 285.36s (at 09:58)
[INFO ]  =================================================================
[INFO ]  Installing kernel headers
[EXTRA]    Installing kernel headers
[EXTRA]    Checking installed headers
[INFO ]  Installing kernel headers: done in 6.47s (at 10:04)
[INFO ]  =================================================================
[INFO ]  Installing C library headers & start files
[INFO ]    =================================================================
[INFO ]    Building for multilib 1/1: ''
[EXTRA]      Configuring C library
[EXTRA]      Installing C library headers
[EXTRA]      Installing C library start files
[INFO ]    Building for multilib 1/1: '': done in 7.65s (at 10:12)
[INFO ]  Installing C library headers & start files: done in 7.72s (at 10:12)
[INFO ]  =================================================================
[INFO ]  Installing pass-2 core C gcc compiler
[EXTRA]    Configuring core C gcc compiler
[EXTRA]    Building gcc
[EXTRA]    Installing gcc
[EXTRA]    Housekeeping for core gcc compiler
[EXTRA]       '' --> lib (gcc)   lib (os)
[INFO ]  Installing pass-2 core C gcc compiler: done in 354.87s (at 16:07)
[INFO ]  =================================================================
[INFO ]  Installing C library
[INFO ]    =================================================================
[INFO ]    Building for multilib 1/1: ''
[EXTRA]      Cleaning up start files
[EXTRA]      Configuring C library
[EXTRA]      Building C library
[EXTRA]      Configuring C library
[EXTRA]      Building C library
[EXTRA]      Installing C library
[INFO ]    Building for multilib 1/1: '': done in 184.69s (at 19:12)
[INFO ]  Installing C library: done in 184.78s (at 19:12)
[INFO ]  =================================================================
[INFO ]  Installing final gcc compiler
[EXTRA]    Configuring final gcc compiler
[EXTRA]    Building final gcc compiler
[EXTRA]    Installing final gcc compiler
[EXTRA]    Housekeeping for final gcc compiler
[EXTRA]       '' --> lib (gcc)   lib (os)
[INFO ]  Installing final gcc compiler: done in 339.13s (at 24:51)
[INFO ]  =================================================================
[INFO ]  Finalizing the toolchain's directory
[INFO ]    Stripping all toolchain executables
[EXTRA]    Installing the populate helper
[EXTRA]    Installing a cross-ldd helper
[EXTRA]    Creating toolchain aliases
[EXTRA]    Removing installed documentation
[EXTRA]    Collect license information from: ~/Crosscompile/.build/arm-rpi-linux-gnueabihf/src
[EXTRA]    Put the license information to: ~/....../RPi/share/licenses
[INFO ]  Finalizing the toolchain's directory: done in 1.80s (at 24:52)
[INFO ]  Build completed at 20190906.020619
[INFO ]  (elapsed: 24:51.86)
[INFO ]  Finishing installation (may take a few seconds)...
[24:52] / %                                                                                                                                                              longervision-GT72-6QE%

This process may take a while. I’m going to sleep tonight. Continue tomorrow…

Alright, let’s continue today. Glad to know it’s built successfully.

Let’s FIRST take a look at what’s under the current folder.

1
2
longervision-GT72-6QE% ls
build.log  rpi.config

And then, let’s take a look at what’s built under the specified destination folder.

1
2
3
4
5
6
7
8
9
10
longervision-GT72-6QE% ls ~/....../CrossCompile/RPi
arm-rpi-linux-gnueabihf  bin  build.log.bz2  include  lib  libexec  share
longervision-GT72-6QE% ls ~/....../CrossCompile/RPi/bin
arm-rpi-linux-gnueabihf-addr2line  arm-rpi-linux-gnueabihf-cpp           arm-rpi-linux-gnueabihf-gcc-ar      arm-rpi-linux-gnueabihf-gprof    arm-rpi-linux-gnueabihf-objdump   arm-rpi-linux-gnueabihf-strip
arm-rpi-linux-gnueabihf-ar         arm-rpi-linux-gnueabihf-ct-ng.config  arm-rpi-linux-gnueabihf-gcc-nm      arm-rpi-linux-gnueabihf-ld       arm-rpi-linux-gnueabihf-populate
arm-rpi-linux-gnueabihf-as         arm-rpi-linux-gnueabihf-elfedit       arm-rpi-linux-gnueabihf-gcc-ranlib  arm-rpi-linux-gnueabihf-ld.bfd   arm-rpi-linux-gnueabihf-ranlib
arm-rpi-linux-gnueabihf-c++        arm-rpi-linux-gnueabihf-g++           arm-rpi-linux-gnueabihf-gcov        arm-rpi-linux-gnueabihf-ldd      arm-rpi-linux-gnueabihf-readelf
arm-rpi-linux-gnueabihf-cc         arm-rpi-linux-gnueabihf-gcc           arm-rpi-linux-gnueabihf-gcov-dump   arm-rpi-linux-gnueabihf-nm       arm-rpi-linux-gnueabihf-size
arm-rpi-linux-gnueabihf-c++filt    arm-rpi-linux-gnueabihf-gcc-8.3.0     arm-rpi-linux-gnueabihf-gcov-tool   arm-rpi-linux-gnueabihf-objcopy  arm-rpi-linux-gnueabihf-strings
longervision-GT72-6QE%

Finally, let’s take a look at the version of our built cross compilers for Raspberry Pi 3B.

1
2
3
4
5
6
7
8
9
10
11
12
13
longervision-GT72-6QE% arm-rpi-linux-gnueabihf-gcc --version
zsh: command not found: arm-rpi-linux-gnueabihf-gcc
longervision-GT72-6QE% ./arm-rpi-linux-gnueabihf-gcc --version
arm-rpi-linux-gnueabihf-gcc (crosstool-NG 1.24.0.6-afaf7b9) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

longervision-GT72-6QE% ./arm-rpi-linux-gnueabihf-g++ --version
arm-rpi-linux-gnueabihf-g++ (crosstool-NG 1.24.0.6-afaf7b9) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Additional issue: It seems current crosstool-NG does NOT officially support Python. Please refer to my issue at crosstool-NG

4. Compile/Build a Package with Generated Toolchain

For most of the packages nowadays, they are

  • either supported by make: ./configure -> make -> make install
  • or supported by CMake: mkdir build -> cd build -> ccmake ../ -> make -> make install

How to use our generated toolchain to compile/build our target packages?

Today, we’re going to take the package flann as an example, which are to be built with CMake.

4.1 CMakeLists.txt Modification

4.1.1 CMake Toolchains - Cross Compiling for Linux

By following CMake Toolchains - Cross Compiling for Linux, we FIRST modify flann CMakeLists.txt a bit by adding the following lines before project(flann).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
cmake_minimum_required(VERSION 2.6)

if(COMMAND cmake_policy)
    cmake_policy(SET CMP0003 NEW)
endif(COMMAND cmake_policy)

set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR arm)

set(CMAKE_SYSROOT ....../arm-rpi-linux-gnueabihf/sysroot)
set(CMAKE_STAGING_PREFIX ....../stage)

set(tools /opt/compilers/RPi)
set(CMAKE_C_COMPILER ${tools}/bin/arm-rpi-linux-gnueabihf-gcc)
set(CMAKE_CXX_COMPILER ${tools}/bin/arm-rpi-linux-gnueabihf-g++)

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

set(CMAKE_TRY_COMPILE_TARGET_TYPE "STATIC_LIBRARY")

project(flann)
string(TOLOWER ${PROJECT_NAME} PROJECT_NAME_LOWER)
  • CMAKE_SYSROOT specify the sysroot directory which emulate your target environment, here, Raspberry Pi 3B
  • CMAKE_STAGING_PREFIX is where the STAGING results store, for the reason that final results may require to be built in multiple stages. You may refer to Linux From Scratch for further background knowledge about that.
  • tools actually specify the building tool directory. Making sure all generated cross compiling tools are under folder ${tools}/bin.

4.1.2 Ignore hdf5

In addition, for the emulated Raspberry Pi 3B sysroot, hdf5 is NOT supported. Therefore, let’s simply comment out the following line in flann CMakeLists.txt.

1
#find_hdf5()

4.2 Cross Compile

Now, let’s start cross-compiling flann.

1
2
3
longervision-GT72-6QE% mkdir build
longervision-GT72-6QE% cd build
longervision-GT72-6QE% ccmake ../

and press c and then t, you’ll see the cross compiling toolchains have been automatically configured as:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
BUILD_CUDA_LIB                  *OFF
BUILD_C_BINDINGS                *ON
BUILD_DOC                       *OFF
BUILD_EXAMPLES                  *OFF
BUILD_MATLAB_BINDINGS           *OFF
BUILD_PYTHON_BINDINGS           *ON
BUILD_TESTS                     *OFF
CMAKE_AR                        */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ar
CMAKE_BUILD_TYPE                *RelWithDebInfo
CMAKE_COLOR_MAKEFILE            *ON
CMAKE_CXX_COMPILER_AR           */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ar
CMAKE_CXX_COMPILER_RANLIB       */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ranlib
CMAKE_CXX_FLAGS                 *-I/usr/include
CMAKE_CXX_FLAGS_DEBUG           *-g
CMAKE_CXX_FLAGS_MINSIZEREL      *-Os -DNDEBUG
CMAKE_CXX_FLAGS_RELEASE         *-O3 -DNDEBUG
CMAKE_CXX_FLAGS_RELWITHDEBINFO  *-O2 -g -DNDEBUG
CMAKE_C_COMPILER_AR             */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ar
CMAKE_C_COMPILER_RANLIB         */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ranlib
CMAKE_C_FLAGS                   *-I/usr/include
CMAKE_C_FLAGS_DEBUG             *-g
CMAKE_C_FLAGS_MINSIZEREL        *-Os -DNDEBUG
CMAKE_C_FLAGS_RELEASE           *-O3 -DNDEBUG
CMAKE_C_FLAGS_RELWITHDEBINFO    *-O2 -g -DNDEBUG
CMAKE_EXE_LINKER_FLAGS          *
CMAKE_EXE_LINKER_FLAGS_DEBUG    *
CMAKE_EXE_LINKER_FLAGS_MINSIZE  *  
CMAKE_EXE_LINKER_FLAGS_RELEASE  *     
CMAKE_EXE_LINKER_FLAGS_RELWITH  *
CMAKE_EXPORT_COMPILE_COMMANDS   *OFF
CMAKE_INSTALL_PREFIX            */usr/local
CMAKE_LINKER                    */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ld
CMAKE_MAKE_PROGRAM              */usr/bin/make
CMAKE_MODULE_LINKER_FLAGS       *    
CMAKE_MODULE_LINKER_FLAGS_DEBU  *  
CMAKE_MODULE_LINKER_FLAGS_MINS  *   
CMAKE_MODULE_LINKER_FLAGS_RELE  *    
CMAKE_MODULE_LINKER_FLAGS_RELW  * 
CMAKE_NM                        */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-nm
CMAKE_OBJCOPY                   */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-objcopy
CMAKE_OBJDUMP                   */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-objdump
CMAKE_RANLIB                    */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ranlib
CMAKE_SHARED_LINKER_FLAGS       *    
CMAKE_SHARED_LINKER_FLAGS_DEBU  *
CMAKE_SHARED_LINKER_FLAGS_MINS  *   
CMAKE_SHARED_LINKER_FLAGS_RELE  *
CMAKE_SHARED_LINKER_FLAGS_RELW  *   
CMAKE_SKIP_INSTALL_RPATH        *OFF
CMAKE_SKIP_RPATH                *OFF
CMAKE_STATIC_LINKER_FLAGS       *
CMAKE_STATIC_LINKER_FLAGS_DEBU  *
CMAKE_STATIC_LINKER_FLAGS_MINS  * 
CMAKE_STATIC_LINKER_FLAGS_RELE  *
CMAKE_STATIC_LINKER_FLAGS_RELW  *
CMAKE_STRIP                     */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-strip
CMAKE_VERBOSE_MAKEFILE          *ON
CPACK_SOURCE_RPM                *OFF
CPACK_SOURCE_TBZ2               *ON
CPACK_SOURCE_TGZ                *ON
CPACK_SOURCE_TXZ                *ON 
CPACK_SOURCE_TZ                 *ON
CPACK_SOURCE_ZIP                *OFF
DPKG_PROGRAM                    */usr/bin/dpkg
NSIS_PROGRAM                    */usr/bin/makensis
NVCC_COMPILER_BINDIR            *
PACKAGE_MAKER_PROGRAM           *PACKAGE_MAKER_PROGRAM-NOTFOUND
PKG_CONFIG_EXECUTABLE           */usr/bin/pkg-config
PYTHON_EXECUTABLE               */usr/bin/python
RPM_PROGRAM                     *RPM_PROGRAM-NOTFOUND
USE_MPI                         *OFF
USE_OPENMP                      *OFF

The LAST step before make is to modify some of the parameters accordingly, in my case:

  • BUILD_PYTHON_BINDINGS: ON -> OFF
  • CMAKE_CXX_FLAGS: -I/usr/include (for lz4.h from my host Ubuntu 19.04)
  • CMAKE_C_FLAGS: -I/usr/include (for lz4.h from my host Ubuntu 19.04)
  • CMAKE_VERBOSE_MAKEFILE: OFF -> ON
  • CMAKE_INSTALL_PREFIX: /usr/local/ -> under which directory you want to install, which can be IGNORED for now

Now, press c, you’ll see

1
2
CMake Warning at CMakeLists.txt:99 (message):
  hdf5 library not found, some tests will not be run

A warining of missing hdf5 is clearly reasonable and acceptable. Then press g.

Finally, it’s the time for us to cross build flann.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
longervision-GT72-6QE% make -j8
/usr/bin/cmake -S....../flann -B....../flann/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start ....../flann/build/CMakeFiles ....../flann/build/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_s.dir/build.make src/cpp/CMakeFiles/flann_s.dir/depend
make -f src/cpp/CMakeFiles/flann_cpp_s.dir/build.make src/cpp/CMakeFiles/flann_cpp_s.dir/depend
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann_s.dir/DependInfo.cmake --color=
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann_cpp_s.dir/DependInfo.cmake --color=
Scanning dependencies of target flann_cpp_s
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_s.dir/build.make src/cpp/CMakeFiles/flann_s.dir/build
make[2]: Entering directory '....../flann/build'
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_cpp_s.dir/build.make src/cpp/CMakeFiles/flann_cpp_s.dir/build
make[2]: Entering directory '....../flann/build'
[ 12%] Building CXX object src/cpp/CMakeFiles/flann_s.dir/flann/flann.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_STATIC -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG   -Wall -Wno-unknown-pragmas -Wno-unused-function -fPIC -o CMakeFiles/flann_s.dir/flann/flann.cpp.o -c ....../flann/src/cpp/flann/flann.cpp
[ 25%] Building CXX object src/cpp/CMakeFiles/flann_cpp_s.dir/flann/flann_cpp.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_STATIC -DFLANN_USE_CUDA -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG   -Wall -Wno-unknown-pragmas -Wno-unused-function -fPIC -o CMakeFiles/flann_cpp_s.dir/flann/flann_cpp.cpp.o -c ....../flann/src/cpp/flann/flann_cpp.cpp
In file included from ....../flann/src/cpp/flann/util/params.h:33,
                 from ....../flann/src/cpp/flann/flann.hpp:42,
                 from ....../flann/src/cpp/flann/flann.h:603,
                 from ....../flann/src/cpp/flann/flann.cpp:31:
....../flann/src/cpp/flann/util/any.h: In member function 'virtual void flann::anyimpl::big_any_policy<T>::static_delete(void**)':
....../flann/src/cpp/flann/util/any.h:81:9: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
         ^~
....../flann/src/cpp/flann/util/any.h:81:55: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the 'if'
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
                                                       ^
In file included from ....../flann/src/cpp/flann/util/params.h:33,
                 from ....../flann/src/cpp/flann/flann.hpp:42,
                 from ....../flann/src/cpp/flann/flann_cpp.cpp:30:
....../flann/src/cpp/flann/util/any.h: In member function 'virtual void flann::anyimpl::big_any_policy<T>::static_delete(void**)':
....../flann/src/cpp/flann/util/any.h:81:9: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
         ^~
....../flann/src/cpp/flann/util/any.h:81:55: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the 'if'
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
                                                       ^
[ 37%] Linking CXX static library ../../lib/libflann_cpp_s.a
cd ....../flann/build/src/cpp && /usr/bin/cmake -P CMakeFiles/flann_cpp_s.dir/cmake_clean_target.cmake
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann_cpp_s.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ar qc ../../lib/libflann_cpp_s.a  CMakeFiles/flann_cpp_s.dir/flann/flann_cpp.cpp.o
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ranlib ../../lib/libflann_cpp_s.a
make[2]: Leaving directory '....../flann/build'
[ 37%] Built target flann_cpp_s
make -f src/cpp/CMakeFiles/flann_cpp.dir/build.make src/cpp/CMakeFiles/flann_cpp.dir/depend
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann_cpp.dir/DependInfo.cmake --color=
Scanning dependencies of target flann_cpp
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_cpp.dir/build.make src/cpp/CMakeFiles/flann_cpp.dir/build
make[2]: Entering directory '....../flann/build'
[ 50%] Building CXX object src/cpp/CMakeFiles/flann_cpp.dir/empty.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_EXPORTS -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG -fPIC   -Wall -Wno-unknown-pragmas -Wno-unused-function -o CMakeFiles/flann_cpp.dir/empty.cpp.o -c ....../flann/src/cpp/empty.cpp
[ 62%] Linking CXX shared library ../../lib/libflann_cpp.so
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann_cpp.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot -fPIC -I/usr/include -O2 -g -DNDEBUG  -shared -Wl,-soname,libflann_cpp.so.1.9 -o ../../lib/libflann_cpp.so.1.9.1 CMakeFiles/flann_cpp.dir/empty.cpp.o -Wl,-whole-archive ../../lib/libflann_cpp_s.a -Wl,-no-whole-archive 
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_symlink_library ../../lib/libflann_cpp.so.1.9.1 ../../lib/libflann_cpp.so.1.9 ../../lib/libflann_cpp.so
make[2]: Leaving directory '....../flann/build'
[ 62%] Built target flann_cpp
In file included from ....../flann/src/cpp/flann/algorithms/all_indices.h:35,
                 from ....../flann/src/cpp/flann/flann.hpp:45,
                 from ....../flann/src/cpp/flann/flann.h:603,
                 from ....../flann/src/cpp/flann/flann.cpp:31:

VERY Important:

  • If you build flann from source on a Raspberry Pi 3B, your system is going to hang HERE, possibly due to lack of memory..
  • Raspberry Pi 3B has 4 cores ONLY, but now you can use MORE cores on your host PC, which is clearly one advantage of using cross compiling.

Finally, you’ll see flann has been successfully cross compiled as:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
....../flann/src/cpp/flann/algorithms/autotuned_index.h: In member function 'void flann::AutotunedIndex<Distance>::optimizeKMeans(std::vector<flann::AutotunedIndex<Distance>::CostData>&) [with Distance = flann::L2<double>]':
....../flann/src/cpp/flann/algorithms/autotuned_index.h:379:31: note: parameter passing for argument of type 'flann::L2<double>' changed in GCC 7.1
         KMeansIndex<Distance> kmeans(sampledDataset_, cost.params, distance_);
                               ^~~~~~
[ 75%] Linking CXX static library ../../lib/libflann_s.a
cd ....../flann/build/src/cpp && /usr/bin/cmake -P CMakeFiles/flann_s.dir/cmake_clean_target.cmake
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann_s.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ar qc ../../lib/libflann_s.a  CMakeFiles/flann_s.dir/flann/flann.cpp.o
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ranlib ../../lib/libflann_s.a
make[2]: Leaving directory '....../flann/build'
[ 75%] Built target flann_s
make -f src/cpp/CMakeFiles/flann.dir/build.make src/cpp/CMakeFiles/flann.dir/depend
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann.dir/DependInfo.cmake --color=
Scanning dependencies of target flann
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann.dir/build.make src/cpp/CMakeFiles/flann.dir/build
make[2]: Entering directory '....../flann/build'
[ 87%] Building CXX object src/cpp/CMakeFiles/flann.dir/empty.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_EXPORTS -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG -fPIC   -Wall -Wno-unknown-pragmas -Wno-unused-function -o CMakeFiles/flann.dir/empty.cpp.o -c ....../flann/src/cpp/empty.cpp
[100%] Linking CXX shared library ../../lib/libflann.so
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot -fPIC -I/usr/include -O2 -g -DNDEBUG  -shared -Wl,-soname,libflann.so.1.9 -o ../../lib/libflann.so.1.9.1 CMakeFiles/flann.dir/empty.cpp.o -Wl,-whole-archive ../../lib/libflann_s.a -Wl,-no-whole-archive -Wl,--push-state,--no-as-needed -Wl,--pop-state 
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_symlink_library ../../lib/libflann.so.1.9.1 ../../lib/libflann.so.1.9 ../../lib/libflann.so
make[2]: Leaving directory '....../flann/build'
[100%] Built target flann
make[1]: Leaving directory '....../flann/build'
/usr/bin/cmake -E cmake_progress_start ....../flann/build/CMakeFiles 0

You can now:

  • make install to install the built/generated libraries installed under CMAKE_INSTALL_PREFIX
  • copy and paste the built/generated libraries onto Raspberry Pi 3B and use it directly.

BTW: do NOT forget to install the header files.

5. Additional Issues

Multilib/multiarch seems to be problematic nowadays. Please pay attention to Multilib/multiarch. Some related issues are enumuated as the end of this blog.

Alright, that’s all for today. Let me go to bed. Good bye…

Posted on | Edited on | In ,

Today, we are going to talk about a fabulous project: stereo vision on a zynq-7010 board.

1. VCSBC nano Z-RH-2

1.1 Hardware

We are using a VCSBC nano Z-RH-2 board for today’s experiment. The board adopted looks like the following:

Front Back Connector
Front Back Connector

More detailed specifications, please refer to Vision Components’s official website.

1.2 Software

After you set up a static IP for this Vision Components SBC, it’s pretty straightforward to ssh into the system.

1
2
3
4
5
6
7
8
9
10
11
longervision-GT72-6QE% ssh user@192.168.1.79
user@192.168.1.79's password: 

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Aug 29 18:17:49 2019 from 192.168.1.200
user@VC-Z:~$

Currently, VC provides Linux Kernel 3.14.79.

1
2
user@VC-Z:~$ uname -a
Linux VC-Z 3.14.79-vc-z #2 SMP PREEMPT Wed Apr 24 18:33:06 CEST 2019 armv7l GNU/Linux

And, let’s take a look at the dual ARMv7 CPUs on zynq-7010.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
user@VC-Z:~$ cat /proc/cpuinfo
processor	: 0
model name	: ARMv7 Processor rev 0 (v7l)
BogoMIPS	: 1725.23
Features	: swp half thumb fastmult vfp edsp neon vfpv3 tls vfpd32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x3
CPU part	: 0xc09
CPU revision	: 0

processor	: 1
model name	: ARMv7 Processor rev 0 (v7l)
BogoMIPS	: 1725.23
Features	: swp half thumb fastmult vfp edsp neon vfpv3 tls vfpd32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x3
CPU part	: 0xc09
CPU revision	: 0

Hardware	: Xilinx Zynq Platform
Revision	: 0000
Serial		: 0000000000000000

2. Stereo Vision

Sorry everybody. Today, I ONLY test stereo vision on ARM. I’ll try to figure out how to flash an open source IP of stereo vision onto zynq-7010, or write my own ASAP.

Hmmmmmmmm… It’s better I keep my code in dark???

2.1 Classical Image Pairs

Let’s try out the stereo vision on some .pgm image pairs FIRST.

1
2
3
4
5
user@VC-Z:~/longervision$ ./pgmpair ../images/aloe_left.pgm ../images/aloe_right.pgm 
width= 1282
height= 1110
fps: 0.119696
process: 398651

My GOD… It’s UNBELIEVABLY SLOW.

aloe_left aloe_right
aloe_left aloe_right
aloe_left Stereo aloe_right Stereo
aloe_left stereo aloe_right stereo

2.2 Live Video Pairs

The BEST demo code to test Vision Componenets Stereo Vision is Eclipse_Example_Projects_VC_Z.

2.2.1 imageCaptureTest

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
#./imageCaptureTest 
0


 ----------------------------------------------------------------------- 
  VCCapt @0xbe87153c                                    Sensor Count: 2  
 ----------------------------------------------------------------------- 
                                                                         
                  Camera Type :  NANO Z RH Rev.0                         
                Sensor 0 Type :  E2V_EV76C560ABT_1_0                     
                Sensor 1 Type :  E2V_EV76C560ABT_1_0                     
                                                                         
   Trigger        Source      :  IMM                                     
                                                                         
     ___________________________________________________________________ 
     Sensor[ 0]:         Shutter:     12345 us,  Gain: 0.00              
                                                                         
                   x0, y0       :  (    0,    0)                         
            binX,     binY      :  (    1,    1)                         
        subsmplX, subsmplY, flip:  (    1,    1,0x00)                    
    (img)          dx, dy, pitch:  ( 1280, 1024), 1280                   
                                                                         
    (img) st, ccmp1, ccmp2, type:  0xb6b03080, 0x00000000, 0x00000000, 0 
                                                                         
      Description                                                        
       Sen dx, Sen dy, Sen pitch:   1280, 1024, 1280                     
       Max dx, Max dy, Max pitch:   1280, 1024, 1280                     
    Binning Max (dx, dy), Square:      2,    1,    1                     
    Subsmpl Max (dx, dy), Square:      8,    8,    0                     
       Init       st,ccmp1,ccmp2:  0xb6b03080, 0x00000000, 0x00000000    
       Init  Shutter, Gain, Type:       10000,  0.00, GREY   (0)         
                  FD, Port, Type:  4, 0, E2V_EV76C560ABT_1_0             
                                                                         
      Volatile Description                                               
            requestCaptID, State:          0, IDLE                       
          captId, TimestampTicks:          0,   6476832867277            
                                                                         
      Externals Description                                              
            pos[0-3], angle[0-2]:  [ 0.00, 0.00, 0.00],[  0.0,  0.0]     
                                                                         
     ___________________________________________________________________ 
     Sensor[ 1]:         Shutter:     10000 us,  Gain: 0.00              
                                                                         
                   x0, y0       :  (    0,    0)                         
            binX,     binY      :  (    1,    1)                         
        subsmplX, subsmplY, flip:  (    1,    1,0x00)                    
    (img)          dx, dy, pitch:  ( 1280, 1024), 1280                   
                                                                         
    (img) st, ccmp1, ccmp2, type:  0xb69c2080, 0x00000000, 0x00000000, 0 
                                                                         
      Description                                                        
       Sen dx, Sen dy, Sen pitch:   1280, 1024, 1280                     
       Max dx, Max dy, Max pitch:   1280, 1024, 1280                     
    Binning Max (dx, dy), Square:      2,    1,    1                     
    Subsmpl Max (dx, dy), Square:      8,    8,    0                     
       Init       st,ccmp1,ccmp2:  0xb69c2080, 0x00000000, 0x00000000    
       Init  Shutter, Gain, Type:       10000,  0.00, GREY   (0)         
                  FD, Port, Type:  5, 1, E2V_EV76C560ABT_1_0             
                                                                         
      Volatile Description                                               
            requestCaptID, State:          0, IDLE                       
          captId, TimestampTicks:          0,   6476832867284            
                                                                         
      Externals Description                                              
            pos[0-3], angle[0-2]:  [ 0.00, 0.00, 0.00],[  0.0,  0.0]     
                                                                         
 ----------------------------------------------------------------------- 
......

2.2.2 imageCaptFPS

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
03:41:47[root@VC-Z] /home/user/vc/Eclipse_Example_Projects_VC_Z/imageCaptFPS
#./imageCaptFPS 
dy= 1024  Clock=100000000  Acquisiton&Copy Duration:         28s     780685us  for 1000 Cycles ==  34.745525fps.

dy= 768  Clock=100000000  Acquisiton&Copy Duration:         28s     789899us  for 1000 Cycles ==  34.734402fps.

dy= 640  Clock=100000000  Acquisiton&Copy Duration:         28s     790000us  for 1000 Cycles ==  34.734283fps.

dy= 512  Clock=100000000  Acquisiton&Copy Duration:         28s     790012us  for 1000 Cycles ==  34.734268fps.

dy= 384  Clock=100000000  Acquisiton&Copy Duration:         28s     790005us  for 1000 Cycles ==  34.734280fps.

dy= 256  Clock=100000000  Acquisiton&Copy Duration:         28s     790000us  for 1000 Cycles ==  34.734283fps.

dy= 192  Clock=100000000  Acquisiton&Copy Duration:         28s     790021us  for 1000 Cycles ==  34.734257fps.

dy= 128  Clock=100000000  Acquisiton&Copy Duration:         28s     789984us  for 1000 Cycles ==  34.734303fps.

dy= 64  Clock=100000000  Acquisiton&Copy Duration:         28s     789987us  for 1000 Cycles ==  34.734299fps.

dy= 32  Clock=100000000  Acquisiton&Copy Duration:         28s     789997us  for 1000 Cycles ==  34.734287fps.

dy= 16  Clock=100000000  Acquisiton&Copy Duration:         28s     790017us  for 1000 Cycles ==  34.734264fps.

dy= 8  Clock=100000000  Acquisiton&Copy Duration:         28s     790002us  for 1000 Cycles ==  34.734280fps.

dy= 4  Clock=100000000  Acquisiton&Copy Duration:         28s     789985us  for 1000 Cycles ==  34.734303fps.

dy= 2  Clock=100000000  Acquisiton&Copy Duration:         28s     789997us  for 1000 Cycles ==  34.734287fps.

dy= 1  Clock=100000000  Acquisiton&Copy Duration:         28s     789994us  for 1000 Cycles ==  34.734291fps.

The above 2 examples are directly run on Vision Components’s board without display, for the given cable is of VGA connector, which is ALREADY outdated for many years. Therefore, in order to show the captured image pairs, in the next section, we’ll have to stream the captured data to a host computer, and display the real-time video pairs.

2.3 Stream/Display From Host Computer

Due to Vision Componenets’ Official documentation VCLinux_Getting_Started.pdf, images captured from camera can be displayed on a remote hosting PC.

2.3.1 Eclipse_Example_Projects_VC_Z (Not Preferred)

Eclipse_Example_Projects_VC_Z.zip provides some source code, which displays images captured from camera on the hosting PC’s Eclipse, by adding the camera as a Remote System to Eclipse.
Anyway, by using this method, you need to prepare ALL the following software and packages in advance.

2.3.2 vcimgnetclient.py

Besides the above method, a much more straightforward method is to adopt python script vcimgnetclient.py provided by Vision Components. However, vcimgnetclient.py is ONLY python2 compatible, and Vision Components has NO plan to provide a python3 compatible version of vcimgnetclient.py.

Therefore, the KEY to use the 2nd method is to make vcimgnetclient.py python3 compatible.

2.3.2.1 PyGTK to PyGTK3

1) 2to3
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
longervision-GT72-6QE% 2to3 vcimgnetclient.py
RefactoringTool: Skipping optional fixer: buffer
RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: set_literal
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: Refactored vcimgnetclient.py
--- vcimgnetclient.py	(original)
+++ vcimgnetclient.py	(refactored)
@@ -274,11 +274,11 @@
             if(chk_rgb_true):
                 dy /= 3
                 #img_data_packed = bytearray(chain.from_iterable(izip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
-                img_data_packed = bytearray(chain.from_iterable(zip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
+                img_data_packed = bytearray(chain.from_iterable(list(zip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:]))))
             else:
                 dy /= 1
                 #img_data_packed = bytearray(chain.from_iterable(izip(mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
-            img_data_packed = bytearray(chain.from_iterable(zip(mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
+            img_data_packed = bytearray(chain.from_iterable(list(zip(mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:]))))
             cpb = Gdk.pixbuf_new_from_data(buffer(img_data_packed), Gdk.COLORSPACE_RGB, False, 8, dx, dy, 3*pitch)
             cpb.save(path, "png")
             
@@ -589,16 +589,16 @@
                         dy /= 3
                         mv_img_data = memoryview(img['data'])
                         #img_data_packed = bytearray(chain.from_iterable(izip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
-                        img_data_packed = bytearray(chain.from_iterable(zip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
+                        img_data_packed = bytearray(chain.from_iterable(list(zip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:]))))
                         self.area.window.draw_rgb_image(gc, 0,0, dx,dy, Gdk.RGB_DITHER_NONE, buffer(img_data_packed),3*pitch,0,0)
                     else:
                         if('flscol'==self.how_select):
                             mv_img_data = memoryview(img['data']).tolist()
-                            r_data = map(falsecolor_r,mv_img_data[0 * pitch * dy:])
-                            g_data = map(falsecolor_g,mv_img_data[0 * pitch * dy:])
-                            b_data = map(falsecolor_b,mv_img_data[0 * pitch * dy:])
+                            r_data = list(map(falsecolor_r,mv_img_data[0 * pitch * dy:]))
+                            g_data = list(map(falsecolor_g,mv_img_data[0 * pitch * dy:]))
+                            b_data = list(map(falsecolor_b,mv_img_data[0 * pitch * dy:]))
                             #img_data_packed = bytearray(chain.from_iterable(izip(r_data, g_data, b_data)))
-                            img_data_packed = bytearray(chain.from_iterable(zip(r_data, g_data, b_data)))
+                            img_data_packed = bytearray(chain.from_iterable(list(zip(r_data, g_data, b_data))))
                             self.area.window.draw_rgb_image(gc, 0,0, dx,dy, Gdk.RGB_DITHER_NONE, buffer(img_data_packed),3*pitch,0,0)
                         else: #grey
                             self.area.window.draw_gray_image(gc, 0,0, 
RefactoringTool: Files that need to be modified:
RefactoringTool: vcimgnetclient.py
2) pygi-convert

Please refer to General Porting Tips

1
2
longervision-GT72-6QE% ./pygi-convert.sh vcimgnetclient.py
longervision-GT72-6QE%
3) Try Running
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
longervision-GT72-6QE% python vcimgnetclient.py 
vcimgnetclient.py:330: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "type" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  self.window =  Gtk.Window(Gtk.WindowType.TOPLEVEL)
vcimgnetclient.py:628: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  mfQuit =  Gtk.MenuItem("Quit")
vcimgnetclient.py:629: PyGIDeprecationWarning: Using non GObject arguments for connect_object() is deprecated, use: connect_data(signal, callback, data, connect_flags=GObject.ConnectFlags.SWAPPED)
  mfQuit.connect_object("activate", lambda a: Gtk.main_quit(), "mfQuit")
vcimgnetclient.py:637: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  bmf = Gtk.MenuItem("File")
vcimgnetclient.py:646: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  mhAbout =  Gtk.MenuItem("About")
vcimgnetclient.py:654: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  bmh = Gtk.MenuItem("Help")
Traceback (most recent call last):
  File "vcimgnetclient.py", line 812, in <module>
    Main()
  File "vcimgnetclient.py", line 719, in __init__
    Frontend.__init__(self, self.img_access)
  File "vcimgnetclient.py", line 301, in __init__
    self.winMain = self.WinMain(arg_img_access, self.dialog)
  File "vcimgnetclient.py", line 341, in __init__
    vbox.pack_start(self.menu_bar, False, False)
TypeError: Gtk.Box.pack_start() takes exactly 5 arguments (4 given)
4)

For ALL vbox.pack_start, add one parameter 0 at the end for each case. For instance:
vbox.pack_start(self.menu_bar, False, False)
to
vbox.pack_start(self.menu_bar, False, False, 0)

Oh my god … There are STILL SO MANY things to do, in order to make vcimgnetclient.py Python3 compatible. Therefore, I implement my own vcimgnetclient_qt.py based on PyQt5.

2.3.3 Longer Vision’s vcimgnetclient_qt.py

2.3.3.1 Server

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
22:42:22[root@VC-Z] /home/user
#whoami
root

22:42:24[root@VC-Z] /home/user
#vcimgnetsrv &
[1] 896

22:42:30[root@VC-Z] /home/user
#ps -e | grep vcimgnetsrv
  896 pts/0    00:00:00 vcimgnetsrv

22:42:40[root@VC-Z] /home/user
#vctp
Acquisiton&Copy Duration:          4s      39644us  for 100 Cycles ==  24.754656fps.

Acquisiton&Copy Duration:          4s      34709us  for 100 Cycles ==  24.784935fps.
......

2.3.3.2 Client

Sorry, I’m NOT going to show my code, but the performance can be demonstrated as follows:

Longer Vision Technology - Vision Component's Image Grabber - PyQt5

Posted on | Edited on | In ,

Oh-my-god, kaggle competition about diabetis… Hmmm, I have something to do these days… It’s also a GOOD chance for me to learn some medical terms… In this blog, we’re going to try out this kaggle competition - APTOS 2019 Blindness Detection.

1. Pre-processing Diabetic Retinopathy Images

It’s NOT hard for us to locate APTOS [UpdatedV14] Preprocessing- Ben’s & Cropping, which seems to me a professional snippet of code preprocessing diabetic retinopathy images. Let’s just try it out directly.

1.1 Implementation

The implementation is cited directly from APTOS [UpdatedV14] Preprocessing- Ben’s & Cropping, with trivial modifications. You are welcome to download my snippet of code here.

1.2 Show Off Intermediately Processed Images

5 Categories Diabetic Retinopathy Concepts
5 Categories Diabetic Retinopathy Concepts
Original Images in Color Original Images in Gray Scale
Original Images in Color Original Images in Gray Scale
Ben Graham’s Preprocessing for Gray Scale Images Ben Graham’s Preprocessing for Color Images
Ben Graham's Preprocessing for Gray Scale Images Ben Graham's Preprocessing for Color Images
Circle Crop Circle Crop High Resolution
Circle Crop Circle Crop High Resolution
Resized Train Resized Train High Resolution
Resized Train Resized Train High Resolution

Some extentions about diabetic retinopathy are given in the end:

Heatmap Inconsistency
Heatmap Inconsistency